There is gold, just one or two steps off the beaten track, but we can't see it, because we are on rails, only thinking in terms the track affords.
But it's not so easy to go off the rails: you can easily get disoriented and overwhelmed, and once you have found something, it's difficult to verify... and even harder to convince anyone else.
The insane man is only insane when in a world of sane men.
All elements lighter than iron _release_ energy when they fuse together, while iron and anything heavier must _consume_ energy to fuse together.
Carbon is one of the most important elements for life due to its unique chemical properties and immense number of useful molecular arrangements.
The molecule we call water is essential to life as well, as it is incredibly effective, compared to the alternatives, at transporting materials to and from different parts of an organism (among other things ).
And so on...
We basically see the world as a texture of possibilities, and there are different poles in the way that works.
Some people are rigidly conforming or conservative, and they perceive the "possibility vectors" as mostly low in magnitude and pointing in the normal way.
Some people are "lateral thinkers" compared to the status quo and see weird opportunities all around them. If they play the piano they'll always be looking to jazz it up in more and more crazy ways.
Some people would have a lateral perception but a conservative morality; like, they would have some deep gut level understanding of queerness but react in conservative ways out of fear or repulsion.
I suppose that real innovation probably isn't that much due to lateral imaginative personality, but more due to having immersive knowledge enough that you just see way ahead, plus confidence and close-knit support from conspirators.
Steve Jobs still has the best quote on this that I know.
Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn't really DO it, they just SAW something. It seemed obvious to them after a while. That's because they were able to connect experiences they've had and synthesize new things. And the reason they were able to do that was that they've had more experiences or they have thought more about their experiences than other people.
Unfortunately, that's too rare a commodity. A lot of people in our industry haven't had very diverse experiences. So they don't have enough dots to connect, and they end up with very linear solutions without a broad perspective on the problem. The broader one's understanding of the human experience, the better design we will have.
It doesn't mean it can't be done, just that it is likely very difficult, or some other tangential breakthrough must happen first.
Fortunately, some rails are far less well-defined than others, and there are often nuggets of gold found in the foggy ambiguity surrounding such rails.
Making manufacturing business is HARD, harder, or many times harder than a yet another online service.
Growing and scaling such business is also hard, and it does not scale much downward as such (you have to fire workers, and they riot on you, storm your apartment and beat you and your familiy.)
The business is noticeably seasonal and have sharp highs and lows when it comes to consumer goods.
"The next big thing" opportunity like phones and tablets are come and go, they last a maximum of 2 "feast" years.
There are "cash crop" type products like battery banks, chargers, usb flash sticks. It makes profit all the time, but it's not that high.
In between big contracts, or when the factory is busy making products for itself in a feast year, factory owner do not shed capacity, but switch most of if to cash crop product
Then he puts together most talented engineers and gives them task to invent random stuff, and do trial run manufacturing.
Instead of setting up a brand around these new products, they are unceremoniously dumped to trade agents.
If agents will be placing a lot of repeating orders, then, more consideration will be given to such product.
That is "the complicated Chinese strategy" in 9 sentences.
I challenge anyone who's read this to propose a single blue ocean idea here
A digital rubber band is not a "blue ocean idea".
It's connecting multiple different technologies to attack a wide open market space. Think: smoking, drinking, gambling, oversleeping, getting over your ex, overspending, unhealthy eating, etc.
If that isn't blue ocean strategy then I don't know what is.
(Edit because of downvotes: I was serious, this is a blue ocean idea. For example, imagine children with their own ontology of language and experience (not culturally acquired). That's about as blue ocean as you can get. Neural networks are pretty close. Neural news at nine.)
As for neural networks, generative models, genetic algorithms, etc., they can help us optimize on known search spaces but that's about all they can do. It's a big leap from designing simulation-optimized bicycle parts to generating truly new ideas.
Did you read the recent Software 2.0 rant by the Tesla AI guy? https://medium.com/@karpathy/software-2-0-a64152b37c35
Other good sources are The Master Algorithm (ML) or The Brain That Changes Itself (neuroscience), the former surveying general capabilities and limitations of various approaches and the later surveying findings from medicine on just how plastic our sensory inputs and interpretive capacities are through recent research.
Second, he's describing a pretty radical implementation of what amounts to a new form of fuzzy constraint programming using exactly the technique I described: brute-force optimization on a very sophisticated objective function. It's fundamentally statistical, and it's still fundamentally an optimization problem. Neural networks happen to be modular compared to other optimization techniques, and they happen to do a miraculous job at capturing latent higher-order structure in these optimization problems.
The real innovation he's describing is that we now have powerful enough optimizers that it's cheaper, easier, faster, and sometimes more reliable to just fuzzy-optimize problems instead of coding bespoke solutions for them. That's pretty amazing, but suggests absolutely nothing.
Now, a self-modifying neural network might be interesting, that can monitor its own performance and decide when to retrain part of itself, mutate its own architecture, or request new training data. But we definitely are not there yet.
With all that said, there's a much more down-to-earth version of your idea that neural networks can have their own ontology and experiences: that's just an anthropomorphization the latent structure that they capture. This isn't unique to NNs -- it's something researchers have been exploiting for decades. It's only a small stretch to argue that that's what fitting a decision tree or a principal components model is. This is reflected directly in the jargon that these models "learn" a representation of the data. You can extract new representations of reality right now by running HDBSCAN on a data set of your choice. Neural networks just let us scale that up to richer, higher-dimensional problems.
Maybe that's all "experience" really is, consuming data from the world and encoding various representations of that data. But when it comes to generating truly novel ideas, instead of incremental improvements on existing ideas, I'm bearish on how far our current neural network technology can go. Maybe AlphaGo, which apparently invented new Go strategies that no human player had ever thought of, is a counterexample. Or it's the exception that proves the rule. I guess we'll see as computation power continues to improve.
Precisely my point.
Remember things like selfie sticks, electronic cigars, the bloody wifi lightbulbs and kettles, hoverboards, child trackers, consumer grade drones, and tons of other stuff that most people buy without giving any thought to who and how made the original product.
Only a few people, for example Musk, might be able to bet on some idea and (maybe) win. He might want to fly now to Mars, but this way ahead, we are nowhere of this capability.