The ideas I am synthesising are:
An article on creating occular prosthesis I saw on tv this morning,
This article on Brute force Proofs for Math problems:
https://motherboard.vice.com/en_us/article/padnvm/200-terabyte-proof-demonstrates-the-potential-of-brute-force-math
And an article on programmers as craftsmen that I read somewhere back in the past:
Might have been this one: http://manifesto.softwarecraftsmanship.org/
I think that there is a cusp point at which an industry transitions from human skill powered to automation. This could be pure mechanical automation/reproduction or in future AI driven systems. This has happened to any and all manufacturing over the past century. Start with Armoury practice or the "American Method" in weapons manufacturing. This was a simple move from craftsman driven industry to an industry built around a component based design where each component was produced by a specialist. Then the specialist was replaced by a cheaper duplication process. This could involve humans, but it was reduced to a series of steps that could be done on machines. The craftsman who once had to know all the different aspects of the production process for a weapon, then faded into irrelevance. (Until the process was re-discovered and turned into a you-tube series)
Why did this happen? Economics? Yes, the production line is more "efficient" at producing a volume of similar items. There are also emergent phenomena that this production model created that were not possible in the craftsman model.
The invention of the production line model allowed many people to be part of the production that were not previously able to be a "one stop shop" craftsman. People could be good at woodwork but not metal work. They could be specialist at creating screws or making barrels. They did not have to be masters of making barrels, triggers, stocks etc. By decomposing the whole item into components, it allows more people to specialist. Some of these specialties were boring and repetitive and would not be "enough" to keep a person who was a "total" craftsman interested, but it opened up an opportunity for people with varied ability and shorter educations who could then be part of a production line for high quality items. It also allows deeper "specialization" to occur. A person who spends all day producing one type of thing, has the opportunity to get very good at it. If they produce many similar things, such as screws or triggers, they can understand them at a level that is not availible to a craftsman who is trying to be good at all the areas of the design and the skills and tools required to produce it. I call this the "specialization limit".
The second emergent phenomena is the scale of the item design. There are only so many skills, techniques and materials that any one craftsman could learn and invest in. This is what I am calling the "scale limits". But a production line can bring the skills of many specialists to a much larger design. Imagine trying to build a jet engine using the craftsman method. One person learns the skills required to operate a huge machine shop and produces all the parts and components to construct one massive engine. Its possible but it would be a singularly unique person who would pursue the education, experience and focus required to craft this kind of thing with the precision required.
There are lots of other phenomena, such as replacement of people on the production line. Even though we do not like the thought of being replaced, its easier to replace and train a person on a single component production, than it is to replace a craftsman who is doing a whole bunch of different skills and processes. This is often the work of a lifetime.
One more phenomenon that I want to make a point of: The phenomenon of the product complexity growing beyond one persons capacity to either design or make. If you look at the growth of complexity from a craftsman made flintlock rifle, through to something like the M1 Garand, I think the craftsman who made the flintlock could have looked at the Garand and "seen" how it worked and given some time and modern tools, probably made a reasonable copy.... but there is a point where the evolution complexity of a family of products exceeds most craftspeople. Weapons are probably not a good subject to try to make this point as the complexity has not started to grow exponentially. The most complex weapons I can think of off the top of my head could probably still be taken apart and rebuilt by a very competent modern fitter/machinist/electrical engineer... basically because they still do all the repair and service on these systems. Perhaps I should call this the "serviceability limit" (since I'm making shit up...). The point being, that without the servicability limit, the complexity of a system can easily grow to exceed any one persons or teams capacity. Look at operating systems if you want an example. They have evolved way beyond the capacity of anyone to service them.
With modern tooling and processes, there are less and less roles for humans in the production line. Only in factories where they are doing small run or are too poor to afford robots do we still find people doing lots of the roles in production. We currently have the robotic technology to replace just about everyone in a production line, however, there are still a range of "hard" bits that no one has bothered to automate. If you look at a current generation vehicle assembly plant (watch any of the "mega-factory" type documentaries, any of them could be fully automated. However, its currently cheaper to use humans to do the "hard" bits than it is to finally automate the whole thing. They main role for humans is still the "creative" bit. Where the product is designed and problems (created by the human users, service agents and consumers of the product) are solved. The funny thing is that most of these creative solutions are pretty common and could be automated in part or in full already. Think about a car assembly plant. The design of a car is not that tricky. They are all essentially variations on a theme. The main variance is the "problem" that they are solving. "I need a small car to run around town and take the kids to school". Not that hard to solve apparently. The whole design could be automated with enough effort, and we could produce thousands of variants of the "small car" design. (With 3D printing of components this has got to be even closer. I'm waiting for the kiosk where I can order a custom car and it will be delivered from some mega-factory that is essentially just a giant car printer and assembler robot. )
The last phenomena is what I call the "forgetting of skills". Once a skill set no longer has a commercial purpose, it becomes a "hobby" at best or is simply forgotten by the majority of people. When I went to school, woodwork and metalwork were still part of the curriculum. Even though there were virtually no manufacturing jobs in the town that would have employed me with them. There is still carpentry in building houses and lots of small fabrication shops doing repairs for all sorts of equipment... but the reality is that these are more foundational skills or hobby skills. I really enjoy wood and metal work but I respect that they are less and less involved in the economy around me. I have to face the fact that I should not encourage my children to study these subjects at school as they are probably irrelevant to them being able to survive. My point is, once a skill set, such as crafting a flintlock rifle is no longer economically valuable, everyone moves on, and the unique set of skills and knowledge that was previously encoded in a single group of craftspeople dies out.
Anyway, to bring this back to my point....
Software development easily falls into the craftsman model at the moment. Even on the big teams where they have compartmentalized design, programming, testing, deployment etc... its only the first step away from "one programmer shops" producing application packages. (Yes, I 'm a one programmer shop and I'm in the dark ages.... and often feel far from being a craftsman... different rant)
My point is, that there are components (clearly I use SDKs and UI controls and frameworks etc) that mean I am not re-inventing the wheel, but each of them has been hand crafted by one or more people. There is very little that could be described as automated in software development. I have used a couple of code generators (shout out to XSD xml parser generator) that are great at doing one thing well. These are the future. Reminds me of an automated UI generator that turned up in a news feed a month or two ago. (Can't remember the reference at the moment) but that is the way of the future. I think we are the last generation of craftspeople programmers.
I think there will be a point not too far in the future where round trip design/programming/compiling/services systems take over software production. Humans will cease to be up to their elbows in every line of code and we will at least be able to describe the interface for a module and any side effects and not only have it written auto magically, but it proven to be correct. This may include a suite of unit tests and any other tests, but these are human artifacts to test for human errors. I think the tests will move to a higher level of abstraction to prove that the design layer is robust, as that is where the creativity will remain for a while. Once the AI is suitable to replace the creative limit of the humans, then the whole process will be automated and software will be a dead art from the olden-days.
So, lets recap:
The "Specialization limit", the "serviceability limit" and the "scale limit". These are all human limits imposed on software systems.
I would argue that the "servicability limit" is very loose in software, as a great deal of it is both blackboxed and so poorly debugged that its hard to argue that any of it is servicable. (Even the stuff that I try to write) Once its compiled and running in the wild on operating systems that have been updated beyond what it was tested on... all bets are off. There is virtually nothing the user can do to "service" the item. It either works or it doesn't.
My ability to service it is still viable but getting harder with the proliferation of platforms. I am working on an old C++ application at the moment. It has fairly clearly defined platform targets that evolve relatively slowly (I say relatively... whole other rant) in comparison with a javascript app that I was working on previously that is just a madhouse of platform variations and patches and just...
At a certain point, it exceeds any one persons ability to service this kind of product. I have a fighting chance as the original designer and builder, but for many legacy systems, the complexity rapidly exceeds the skills of anyone who inherits the system. Keeping up to date with platform movements and maintaining a heritage skill set and knowledge base to understand legacy codebases gets pretty ugly. Just in my own work, I am now looking backward over more than a decade of development using a slew of languages, frameworks and sdks, along with a whole pile of bad design choices and trying to decide if its worth trying to modernise the program. (not that it had a large user base but it represents a big chunk of my life that I am not sure I want to say goodbye to yet...)
The "specialisation limit" is also starting to kick in. I actively work in a range of languages across a few platforms and there is a point beyond which I cannot work in any more. Trying to pick up something that I used to be good at and get back into it, takes some work. I notice the cost. Maybe it would have been easier when I was young and my head was empty, but its filled with all sorts of knowledge that is no longer relevant for all sorts of systems that no one cares about anymore. (I think I need to burn my bookshelf.... )
Watching the library throw out books that I have read is a pretty shocking thing.... especially when you look inside the cover and see that it was checked out exactly 5 times in its lifespan. But the point is, these skill sets are dead. No one cares anymore. They are not economically valuable and my children will never learn them.
The "scale limit" is an interesting issue with software, in that without automated tools, there is a point beyond which we cannot go with a codebase. Its just too complex to comprehend in its entirety. (This is without taking into account the hidden complexity of all the SDK's and the platform code that its running on...) Some of the round trip type tools allow you to deal with larger amounts of complexity, but at a certain point, it will exceed your limits, simply because we all have limits. Automated systems are theoretically limited but the reality is that they can run with much larger capacity than any single human... and at that point we are irrelevant. We will no longer be ecconomically viable.
Programing will be a hobby at best. Our children will order software from automated kiosks that present customized packages assembled by software robots. The company with the best robots will win.
It's interesting to look at the evolution of machine tools along with the evolution of the production line. Mechanical tools, Electric tools, CNC tools, lazer tools, 3D printers, design software, welding robots etc.
Looking at a woodworking shop (cause I can watch youtube..) you can contrast hand tools, planes, chisels, saws etc. with power tools, power plane, thicknesses, joiner, mortiser, biscuit joiner, table saw etc. The advantage of the power tools is speed and reproduce-ability. The material is the same and the construction is the same, just faster.
Once you move up to CNC driven cutters and routers or 3D printers, you are no longer working the same material or construction methods. Essentially, the materials have had to evolve to suit the tools, and the products that are produced are no longer the same. Take a wooden box made with traditional techniques vs a wooden box made of MDF cut out on a CNC router, assembled by a robot and finished in a spray booth. The first has been crafted from peices of natural wood, with grain and form. Finished in a pleasing way by the maker. If the maker used hand tools, it may be a little uneven or not, the difference between handtools used with care and power tools is not that great. The difference between that product and the one produced by the cutting robot however is more distinct.
Both are functionally similar, in that they are a box, but the second is really defined by the process and materials. This is because the production line for robotic wooded products is still only in about the second or third generation. The next generation will be robotics that works with natural materials and reproduces a wooden box that appears to be made by a craftsman. Initially there will be all sorts of cheats to make it easier for the robot to handle the components and to cut the joints... but these are not hard to overcome. It's quite easy to visualize a production line of robots manufacturing wooden boxes that are not easy to differentiate from one made by a human craftsman.
All these tools are working in the production space. None have yet moved into the "solving a problem" meta space. A robot has not yet been developed that can identify the need for a wooden box, build it (or commission it) and supply it/install it and then finally service it through its functional life. This would be a vertically integrated system.
If we looked at software developers tools... its hard to see that we are really that far along.
Text editors are pretty nice, but once you start to list all the "power tools" that can be used, the list is quiet short and is only implemented in a few languages.
- Indenting / formatting tools - pretty print the file, de-mimify
- Spell checkers - syntax checkers and variable/function name checkers
- Cross referencing tools - quickly jump between files, declarations, definitions etc
- Re-factoring tools - lots
In the rest of the toolchain, we have testing tools, compliers, linkers, packaging macros, build tools, installer builders etc. Let's not belabor the point...
Every one of these is still just duplicating what you could do by hand. (Yeah, I realize that compiling a program by hand is beyond me...) but that's the point. These tools are simply doing what is humanly possible... faster. Essentially, "powertools" in the above woodwork analogy. None of them is doing anything not humanly possible... the materials are still the same. There is little integration between the systems (except the compiler and linker... again not the point)
I think the next step will be when the materials are modified to be more machine friendly than human friendly. Looking at all the "intermediate code" and java bytecode that is around, you can see the process is clearly in process. While we could kind of work with bytecode, its not economically viable. There is little point in trying and so the skills are already being forgotten (or not taught to the next generation). I find it hard to imagine that I would encourage my kids to learn assembler. It's just not a good use of their time.
When the software design tools do not bother to produce human readable code, even as an intermediate step, programmers are irrelevant. When the design tools take a rough problem description, then software architects are irrelevant. When bugs are automatically removed, then software serviceability will be solved.
I think that this will happen when we have a high enough level language to describe a problem and its boundaries and a compiler/software robot can translate that into a solution and customize it to our desire.
With the current crop of AI, I think the question is whether this will be a human readable language or simply a pattern recognition neural net that can interpret our vague problem definitions and produce a software solution on demand. At which point, there will be little point in evolving operating systems by human hands anymore. Machines do not need any of the graphical user interface crap that has bloated most of our OS's.
Anyway, the vertical integration process will continue.
It's interesting that consumer goods like the above "wooden box" example are still so far from being vertically integrated and human free.
There is no robot or chain of robots that can harvest a wild grown tree, mill it, dry it, cut, fit and assemble it from real wood into a box and deliver it to me. Note, wild grown tree and real wood.
There are sections of that production chain that exist and have been integrated, but they are still little autonomous units that are struggling to connect.
There are a few more production lines that are able to consume wood chips, turn them into MDF/plywood etc. and cut, assemble and package for delivery, ship it and deliver it... but sill not even close to vertically integrated if you look at the "needs" end of the chain.
For me vertically integrated would be to start with a design (or brain fart) from me in a natural language including the type of wood, figure, finish etc (or even have that automagically recommended (not hard based on my preferences)) all the customization with my name, personal carving styles, size, inserts etc, go out, source the materials (cut on demand or stockpiled by robots) manufacturing the box, pack it, ship it to me and present it in a useful time frame.
Bringing all that together will be the work of the next few generations. I can see it happening with engineered materials in the production section of the chain. But the problem definition is still rudimentary with our current natural language processing, the automated design should be pretty straght forward, but isnt yet. The packaging, shipping and delivery is still at the whim of retailers who are struggling to produce a pair of customised pants. Once you introduce the additional problems of non-engineered materials, its not hugely more complex, but the robotics need to be a lot better.
It would be awesome to see a tree farm managed by robots. Maybe not "managed" but certainly operated.... I predict in the next 20 years.
Having finished this brain fart I then read... https://www.microsoft.com/en-us/research/blog/program-repairs-programs-achieve-78-3-percent-precision-automated-program-repair/
