I recently had a look at a site/app called Kaggle. (www.kaggle.com.au) and have come to the conclusion that crowd sourcing can be used for evil.
This is not specifically an exercise in Kaggle bashing... I have also formed an intense dislike of a number of other crowd source systems.
My fundamental dislike is that it takes a few hundred highly trained people and all their time and rewards a single person/team a few hundred bucks while on the other side of the equation, providing the "sponsor" with a solution that can be leveraged for significant gain.
This simply turns the value of all that labor and knowledge into virtually a worthless commodity. Each person had to pay dearly in time and money for the education to even get into the competition, they then had to spend the time, labor and equipment resources to actually compete and then (mostly) did not get any material reward. To add insult to injury, the sponsor gets to keep the solution all for themselves. No public good what so ever.
The worst problem is that this kind of system provides a fairly deterministic result for the "sponsor". In that the vast majority of the challenges result in a solution (where the data set actually has a useful solution) this means that this is a viable and insanely cheap way to do R&D rather than employ a bunch of knowledge workers and finance all the failures, you only need to pay a pittance for the best solution from a buffet of options. SCORE!!!!.
Suddenly using something like Kaggle is soooooo much more compelling than actually employing all those highly trained but expensive knowledge workers that the western economies have been hoping like hell would start to pay the bills after their tax base all went to the developing world. DUH!
So.... not only does it suck to be a knowledge worker in a country that has no R&D sector... but it sucks to be a knowledge worker anywhere. You now have to "compete" in a game to try to "win" a few cents on the dollar for your labor.... lulz.
There is a shiny inner core to this cloud of choking gunk... the problems that are being solved and the solutions have two nasty stings in the tail for a company hoping to turn them into a product... the first is implementation and the second is .... maintenance.
So while a company can skip the majority of the R&D cost and time required to get a solution, they still need people to product-ise it and keep the system running. At some point they will have to hire someone somewhere who cares enough about the product to actually do some work.
If they out-source everything their product will only look good on paper. There will be no support, no in-house knowledge and no way to evolve. The question is what sort of clock-speed this realisation will have. If the crowd sourcing model destroys the knowledge economy before the downside of crowd sourcing catches up with the head of the cycle, will there be any knowledge economy left?
Wednesday, April 27, 2011
Yet another Zombie metaphore
http://www.escapistmagazine.com/articles/view/issues/issue_303/8816-What-Purpose-Minecraft-Zombies
I like this article. Still trying to articulate why.
I like this article. Still trying to articulate why.
Zombie training for Post-appocalyptic narrative crafting
http://www.escapistmagazine.com/articles/view/issues/issue_303/8818-How-Games-Get-Zombies-Wrong
This is a pointed article about Zombies and their narrative function. It ties into my thinking on post-appocalyptic narratives.
Zombies represent death personified. They are just faster and more obvious than all the other things that can kill you in an environment stripped of the modern luxuries like healthcare, farming, law, running water etc.
Its hard to make a narrative work that uses a slow decay into hunger and desperation. Its boring and tedious and hard to convey to the reader/viewer.
Its easier to user a device like a zombie horde to represent the inevitable surprises and the effects of fear and paranoia on survivors. Zombies give a visual, physical reality that has the same effect as the unseen forces that can turn a survivor or group of survivors into a self-destructive mob and pretty soon after... roadkill.
Try watching a movie like "The Road". While its an incredibly moving experience and rates in my top 10 of post-apocalyptic movies; its still trying to convey subtle emotional transformations that are just difficult to communicate due to the limits of the medium. The sense of time and the effects of that time on the characters in the movie is limited. The narrative has an imperative to keep moving and keep the viewer engaged. Its impossible to take a long period of screen time, to convey a long period of narrative time. The viewers, reviewers and execs will not or cannot take that journey.
So how can a writer/producer communicate these difficult issues. Perhaps not verbatim, rather by creating a narrative artifice such as a Zombie horde. The horde acts to poison and destroy everything that previously supported and nurtured the characters. So rather than having a fine old time living in a city full of resources ( such as in "Tommorow when the world ended") the characters are forced to avoid cities and any places that seem inhabited or habitable. The comfort of strangers suddenly has the obvious risk of the stranger turning on the character. Seeing a figure moving in the distance becomes the cue to run in the opposite direction rather than move in and say "Hi". Zombies are a narrative artifice that wraps all of these difficult concepts up into a personification that can be seen and shot-at and run away from. But is, in the end, everywhere. Its only a matter of time.....
This is a pointed article about Zombies and their narrative function. It ties into my thinking on post-appocalyptic narratives.
Zombies represent death personified. They are just faster and more obvious than all the other things that can kill you in an environment stripped of the modern luxuries like healthcare, farming, law, running water etc.
Its hard to make a narrative work that uses a slow decay into hunger and desperation. Its boring and tedious and hard to convey to the reader/viewer.
Its easier to user a device like a zombie horde to represent the inevitable surprises and the effects of fear and paranoia on survivors. Zombies give a visual, physical reality that has the same effect as the unseen forces that can turn a survivor or group of survivors into a self-destructive mob and pretty soon after... roadkill.
Try watching a movie like "The Road". While its an incredibly moving experience and rates in my top 10 of post-apocalyptic movies; its still trying to convey subtle emotional transformations that are just difficult to communicate due to the limits of the medium. The sense of time and the effects of that time on the characters in the movie is limited. The narrative has an imperative to keep moving and keep the viewer engaged. Its impossible to take a long period of screen time, to convey a long period of narrative time. The viewers, reviewers and execs will not or cannot take that journey.
So how can a writer/producer communicate these difficult issues. Perhaps not verbatim, rather by creating a narrative artifice such as a Zombie horde. The horde acts to poison and destroy everything that previously supported and nurtured the characters. So rather than having a fine old time living in a city full of resources ( such as in "Tommorow when the world ended") the characters are forced to avoid cities and any places that seem inhabited or habitable. The comfort of strangers suddenly has the obvious risk of the stranger turning on the character. Seeing a figure moving in the distance becomes the cue to run in the opposite direction rather than move in and say "Hi". Zombies are a narrative artifice that wraps all of these difficult concepts up into a personification that can be seen and shot-at and run away from. But is, in the end, everywhere. Its only a matter of time.....
Fun with matching algorithms
http://www.theregister.co.uk/2011/04/26/amazon_pricing_algorithms/
This is a fun little example of matching algorithms with limited inputs and a very short feedback loop. What do you get.... uh, growth... duh.
This is a fun little example of matching algorithms with limited inputs and a very short feedback loop. What do you get.... uh, growth... duh.
Thursday, April 14, 2011
Whats it all worth?
Debit, Investment, return, cost, price, value, fixed, variable, risk, reward, time...
http://techcrunch.com/2011/04/10/peter-thiel-were-in-a-bubble-and-its-not-the-internet-its-higher-education/
http://www.npr.org/templates/story/story.php?storyId=112432364
The above articles talk about some of the American problems with valuation of higher education. They echo a sentiment that I have been reading about here. It's not as clear and its not the same, but the pattern is showing up. The problem is the idea of giving people an undergraduate degree and assuming its got some sort of intrinsic value. (The fact that you end up in debit for it is just a really nasty side issue)
I have just read another article about problems with the Australian VET sector being squeezed by the uncapping of University places.
http://www.theaustralian.com.au/higher-education/tafe-diplomas-losing-their-vocation/story-e6frgcjx-1226038131581
I think these articles all tap into the same vein of discontent. The fact that getting a good education is becoming less and less a determining factor for your future health, wealth and stability. Like any investment, once the risk goes up and the returns go down, the value of the investment starts to look pretty uncertain.
I like the phrase from the first article of an investment bubble in higher education. My feel is that its already deflating in Australia and the reforms that are busy moving the deckchairs on the titanic are just obscuring the problem.
Before I read these articles the way I was trying to articulate the issue was from the point of view of Universities as generators of wealth. The problem that I was seeing is that the economy has changed from a manufacturing and agriculture base to a resource and services base. China and Germany are dominating the manufacturing while the rest of the developed world tries to take the high ground in a Knowledge Economy, all the while that google et. al are making knowledge freely availible to the whole world.... Nothing could go wrong here.
So my thesis was that Universities are the source of "New" knowledge which previously would have fed down into the local and regional area and feed the renewal of industry. Now we have seen a long depression in the research budgets and the spinoffs of the University sector into Industry, mixed with a fundamental shift in off-shoring anything that can be moved to China. So the dynamic that previously renewed the industry in Australia has dissipated. Universities are finding they are irrelevant and have little to offer. This reinforces the poor value proposition of investing in the research and higher ed sector and the demand for the Universities to generate more graduates with the same or less funding. This is one of those "Oh Shit...." dynamics. Otherwise known as the death spiral.
The problem is that it can't be fixed by throwing money at it. Because every bit of new knowledge generated by an Australian University will be availible to everyone in the world the second its published. So both the initial investment in the research and the flow on effect of commercializing it is being exported at the speed of light via fiber optic cables.
The free market in knowledge is a great leveler.
Currently the economic powerhouses have the capacity to build on new knowledge. China, India, Brazil and weirdly Germany have the capital and the existing industrial base and the cultures to turn knowledge into practical industry. Australia on the other hand has an industrial sector that is choking to death and every functional business is putting downward pressure on wages and off-shoring everything that moves. We have no capacity to grow on. There is no critical mass... and every time something tries to grow it gets off-shored.
This is not an rant against off-shoring. The point is that this cycle will happen to any country where the bubbling vat that transforms knowledge into real value is drained. Australia has had a brain drain, an industry drain, a capital drain and is now living through a period where we have a constant capacity drain. We have no critical mass in either the knowledge side of the equation or the industry side of the equation. There are lots of tiny little companies doing amazing things. What we need is a massive number of companies doing normal boring stuff. This is the fertile ground in which people are trained and get ideas, its the soft landing area when companies and people make mistakes and try innovative ideas. Its the fertile ground in which the seeds of new knowledge can be planted, and just be sheer quantity, find someone who has the interest and opportunity to give them a go. In the best case scenario, a new idea finds a number of niches that it can be tested in. Some work, some fail, and finally one takes off.
We, to extend the metaphor, have a sparsely populated, hostile environment where there is little risk taking behavior and so very few ideas can be tried, fewer get created and we reach stagnation. No University research and no Industry. Welcome to Australia 2011.
The problem is that like any good sinking ship story, the more the rats see other rats jumping off, the more they jump off creating a chain reaction that, funnily enough, this pattern reflects the market dynamics that happened at the time of the GFC. It's essentially a crisis of confidence that ended the bubble ( as all good bubbles do) and the value collapsed back down until everything devalued to the point where the difference between perceived value and "fundamentals" was bleedingly obvious to the consumers and they started to consume again. Then the back swing begins.... pendulum whatever...
The converse view is the "Counter-Cyclical" idea. Run in the opposite direction to the herd, buy when the market is down, sell when its up... and so we come full circle to the first article at the top of the page. The featured guy making the dire predictions ( or stating the obvious if you have a brain) has made his fortune playing the counter-cyclical game.
So what in the end do we have. Cycles that are caused by too many people being on the cycle and not enough being "counter" or playing the opposing strategy.
If you apply this thinking to economics, its basically means that too many people are holding the same opinions in the market about value at the same time. Ditto with education, ditto with housing etc. Its not just that people make stupid value judgements... that's normal... its just that too many other people are making the same value judgements using the same strategy at the same time.
This problem is magnified when you have the effect of collective bargaining in the market place. Lots of people get together and all bet in the same direction, often by proxy. Think "investment manager" or automated fund trading. This allows thousands or millions of people to bet exactly the same, at the same time and cannot help but cause a tiny bubble. When lots of these systems agree, or worse, feed off each others agreement and play follow the leader we have a massive rush of passengers(Lemmings) to one side of the ship. The market gets distorted and the media picks up on the trend .... and magnifies it by only reporting on one side of the story. So everyone who watches the same news at the same time... gets the same herd reaction and tries to run in the same direction.... and so more passengers run to the already crowded side of the ship.... ta dah! We have a problem.
Markets are regulated primarily by chaos. Chaos causes an roughly even distribution across all positions and all possible strategies. Effectively everyone running in their own direction, at their own pace and following whatever strategy they like. The problems happen when order starts to form within the chaos. (This is the sort of thing professional gamblers love, because they can use order to make money) Markets are eventually destroyed by emergent order. The more order, patterns and structures appear in the market, the more trends there are to predict the market, the less distribution there is. Value starts to pile up within the patterns. But value is not a real thing... its the "difference" between "having" and not "having" something. So differences appear all along the edges of the patterns. Then as the patterns move to the new edges, the morph, like a cheap edge detection image filter applied repeatedly. The emergent nature of the patterns would be fine in an abstract numerical space but we are dealing with units and materials that have inherent form. There are regulations and quantity limits in the market place that cause interference in the movement of the patterns through the space. These cause weird pileups and slack areas where value either cannot exist ( due to a flattening effect) or constant value ( a money drain or money tree) Its within this environment of reefs and pools that the chaos should exist. But due to the ordering effect of people playing follow the leader we get bubbles forming.
There is no point trying to manipulate the regulations in the market. All this does is change the environment in which the action happens. We need to reduce the factors that facilitate mass migrations of people to the same or similar strategies. Cause the big secret that no one wants to admit is that a market is a closed game and for some people to be winning, someone has to be loosing. It's simply differences between "having" and "not having". Value (i.e money) is a perception game.
End rant.
http://techcrunch.com/2011/04/10/peter-thiel-were-in-a-bubble-and-its-not-the-internet-its-higher-education/
http://www.npr.org/templates/story/story.php?storyId=112432364
The above articles talk about some of the American problems with valuation of higher education. They echo a sentiment that I have been reading about here. It's not as clear and its not the same, but the pattern is showing up. The problem is the idea of giving people an undergraduate degree and assuming its got some sort of intrinsic value. (The fact that you end up in debit for it is just a really nasty side issue)
I have just read another article about problems with the Australian VET sector being squeezed by the uncapping of University places.
http://www.theaustralian.com.au/higher-education/tafe-diplomas-losing-their-vocation/story-e6frgcjx-1226038131581
I think these articles all tap into the same vein of discontent. The fact that getting a good education is becoming less and less a determining factor for your future health, wealth and stability. Like any investment, once the risk goes up and the returns go down, the value of the investment starts to look pretty uncertain.
I like the phrase from the first article of an investment bubble in higher education. My feel is that its already deflating in Australia and the reforms that are busy moving the deckchairs on the titanic are just obscuring the problem.
Before I read these articles the way I was trying to articulate the issue was from the point of view of Universities as generators of wealth. The problem that I was seeing is that the economy has changed from a manufacturing and agriculture base to a resource and services base. China and Germany are dominating the manufacturing while the rest of the developed world tries to take the high ground in a Knowledge Economy, all the while that google et. al are making knowledge freely availible to the whole world.... Nothing could go wrong here.
So my thesis was that Universities are the source of "New" knowledge which previously would have fed down into the local and regional area and feed the renewal of industry. Now we have seen a long depression in the research budgets and the spinoffs of the University sector into Industry, mixed with a fundamental shift in off-shoring anything that can be moved to China. So the dynamic that previously renewed the industry in Australia has dissipated. Universities are finding they are irrelevant and have little to offer. This reinforces the poor value proposition of investing in the research and higher ed sector and the demand for the Universities to generate more graduates with the same or less funding. This is one of those "Oh Shit...." dynamics. Otherwise known as the death spiral.
The problem is that it can't be fixed by throwing money at it. Because every bit of new knowledge generated by an Australian University will be availible to everyone in the world the second its published. So both the initial investment in the research and the flow on effect of commercializing it is being exported at the speed of light via fiber optic cables.
The free market in knowledge is a great leveler.
Currently the economic powerhouses have the capacity to build on new knowledge. China, India, Brazil and weirdly Germany have the capital and the existing industrial base and the cultures to turn knowledge into practical industry. Australia on the other hand has an industrial sector that is choking to death and every functional business is putting downward pressure on wages and off-shoring everything that moves. We have no capacity to grow on. There is no critical mass... and every time something tries to grow it gets off-shored.
This is not an rant against off-shoring. The point is that this cycle will happen to any country where the bubbling vat that transforms knowledge into real value is drained. Australia has had a brain drain, an industry drain, a capital drain and is now living through a period where we have a constant capacity drain. We have no critical mass in either the knowledge side of the equation or the industry side of the equation. There are lots of tiny little companies doing amazing things. What we need is a massive number of companies doing normal boring stuff. This is the fertile ground in which people are trained and get ideas, its the soft landing area when companies and people make mistakes and try innovative ideas. Its the fertile ground in which the seeds of new knowledge can be planted, and just be sheer quantity, find someone who has the interest and opportunity to give them a go. In the best case scenario, a new idea finds a number of niches that it can be tested in. Some work, some fail, and finally one takes off.
We, to extend the metaphor, have a sparsely populated, hostile environment where there is little risk taking behavior and so very few ideas can be tried, fewer get created and we reach stagnation. No University research and no Industry. Welcome to Australia 2011.
The problem is that like any good sinking ship story, the more the rats see other rats jumping off, the more they jump off creating a chain reaction that, funnily enough, this pattern reflects the market dynamics that happened at the time of the GFC. It's essentially a crisis of confidence that ended the bubble ( as all good bubbles do) and the value collapsed back down until everything devalued to the point where the difference between perceived value and "fundamentals" was bleedingly obvious to the consumers and they started to consume again. Then the back swing begins.... pendulum whatever...
The converse view is the "Counter-Cyclical" idea. Run in the opposite direction to the herd, buy when the market is down, sell when its up... and so we come full circle to the first article at the top of the page. The featured guy making the dire predictions ( or stating the obvious if you have a brain) has made his fortune playing the counter-cyclical game.
So what in the end do we have. Cycles that are caused by too many people being on the cycle and not enough being "counter" or playing the opposing strategy.
If you apply this thinking to economics, its basically means that too many people are holding the same opinions in the market about value at the same time. Ditto with education, ditto with housing etc. Its not just that people make stupid value judgements... that's normal... its just that too many other people are making the same value judgements using the same strategy at the same time.
This problem is magnified when you have the effect of collective bargaining in the market place. Lots of people get together and all bet in the same direction, often by proxy. Think "investment manager" or automated fund trading. This allows thousands or millions of people to bet exactly the same, at the same time and cannot help but cause a tiny bubble. When lots of these systems agree, or worse, feed off each others agreement and play follow the leader we have a massive rush of passengers(Lemmings) to one side of the ship. The market gets distorted and the media picks up on the trend .... and magnifies it by only reporting on one side of the story. So everyone who watches the same news at the same time... gets the same herd reaction and tries to run in the same direction.... and so more passengers run to the already crowded side of the ship.... ta dah! We have a problem.
Markets are regulated primarily by chaos. Chaos causes an roughly even distribution across all positions and all possible strategies. Effectively everyone running in their own direction, at their own pace and following whatever strategy they like. The problems happen when order starts to form within the chaos. (This is the sort of thing professional gamblers love, because they can use order to make money) Markets are eventually destroyed by emergent order. The more order, patterns and structures appear in the market, the more trends there are to predict the market, the less distribution there is. Value starts to pile up within the patterns. But value is not a real thing... its the "difference" between "having" and not "having" something. So differences appear all along the edges of the patterns. Then as the patterns move to the new edges, the morph, like a cheap edge detection image filter applied repeatedly. The emergent nature of the patterns would be fine in an abstract numerical space but we are dealing with units and materials that have inherent form. There are regulations and quantity limits in the market place that cause interference in the movement of the patterns through the space. These cause weird pileups and slack areas where value either cannot exist ( due to a flattening effect) or constant value ( a money drain or money tree) Its within this environment of reefs and pools that the chaos should exist. But due to the ordering effect of people playing follow the leader we get bubbles forming.
There is no point trying to manipulate the regulations in the market. All this does is change the environment in which the action happens. We need to reduce the factors that facilitate mass migrations of people to the same or similar strategies. Cause the big secret that no one wants to admit is that a market is a closed game and for some people to be winning, someone has to be loosing. It's simply differences between "having" and "not having". Value (i.e money) is a perception game.
End rant.
Thursday, April 7, 2011
Power from an unusual source
http://3.14.by/en/read/building-nuclear-reactor-at-home
Interesting hack for powering low power device in remote area for long periods of time.
Interesting hack for powering low power device in remote area for long periods of time.
Multivariate Testing for the Web
http://www.smashingmagazine.com/2011/04/04/multivariate-testing-101-a-scientific-method-of-optimizing-design/
Passable explanation and insight into this technique.
Passable explanation and insight into this technique.
Wednesday, April 6, 2011
Steve Jobs Thoughts on Flash
http://www.apple.com/hotnews/thoughts-on-flash/
Its been around for a while but I've only just found it. This is the best ( getting information from the source is like that!) explanation of the Adobe Flash vs Apple spat. The points are well written and essentially un-arguable. A cynical person might wonder how many people spent time polishing this memo before it was published; but thats the nature of publishing on behalf of a large organisation... so.... back to reality.
Sux to be Adobe.
Its been around for a while but I've only just found it. This is the best ( getting information from the source is like that!) explanation of the Adobe Flash vs Apple spat. The points are well written and essentially un-arguable. A cynical person might wonder how many people spent time polishing this memo before it was published; but thats the nature of publishing on behalf of a large organisation... so.... back to reality.
Sux to be Adobe.
Tuesday, April 5, 2011
Yet another programming gem
http://asserttrue.blogspot.com/2009/03/how-to-write-fast-code.html
Go faster by doing less. Obvious once you conceptualise it and articulate it.
Go faster by doing less. Obvious once you conceptualise it and articulate it.
Monday, April 4, 2011
Master Programmer Post
http://www.zedshaw.com/essays/master_and_expert.html#
This post explains the idea about moving toward simplicity as a property of mastery. I agree.
This post explains the idea about moving toward simplicity as a property of mastery. I agree.
Subscribe to:
Posts (Atom)