http://needforair.com/blog/2012/04/18/game-of-go/
Interesting.
Showing posts with label algorithms. Show all posts
Showing posts with label algorithms. Show all posts
Tuesday, May 8, 2012
Wednesday, March 14, 2012
Out of control sales bots
http://carlos.bueno.org/2012/02/bots-seized-control.html
There is so much fun to be had with the content in this article.
The fact that Amazon is now just a giant bot feeding ground, run by bots, and exploited by other bots; is just a little bit scary.
The fact that the Amazon "business" cannot be "stopped" or reset to rid it of these bots suggests that its really close to being a permanent "environment". As long as the bots keep paying the bills... who will try to stop it?
The Matrix has you....
There is so much fun to be had with the content in this article.
The fact that Amazon is now just a giant bot feeding ground, run by bots, and exploited by other bots; is just a little bit scary.
The fact that the Amazon "business" cannot be "stopped" or reset to rid it of these bots suggests that its really close to being a permanent "environment". As long as the bots keep paying the bills... who will try to stop it?
The Matrix has you....
Monday, November 7, 2011
Editors + Content Creators = Quality?
http://sparksheet.com/return-of-the-editor-why-human-filters-are-the-future-of-the-web/
This article is interesting and makes a good point. The same point that is being made in academic circles, that peer review and editorial selection is the only way to separate the wheat from the chaff. Without peer review and examination by the community, it becomes increasingly difficult to stay relevant and in touch, let along maintain "accepted" standards of presentation and language.
The contrary argument that always gets trotted out is that this prevents "new" or radical ideas as they tend to be suppressed (overtly or covertly) by "established" ideas. While I think this is a danger, its true in any system that contains "vested interests" or "assumptions". Revolutionary change (great strides) is difficult for any system that is geared toward evolutionary (many small steps) change. This is more a factor of the mechanism of the system than the desire or intent of any one person within the system.
So what about algorithmic quality filters? Do they allow or reject revolutionary content? My guess would be ... it depends! An algorithm is a reflection of the programer(s) who created it. This usually means that it carries all their assumptions and flaws. More simply put, is the algorithm essentially a black list or white list model. Algorithms that are smart enough to do semantic analysis are just not yet feasible on the large scale, but hopefully they will be making an entrance soon and go some way to solving this type of problem.
This article is interesting and makes a good point. The same point that is being made in academic circles, that peer review and editorial selection is the only way to separate the wheat from the chaff. Without peer review and examination by the community, it becomes increasingly difficult to stay relevant and in touch, let along maintain "accepted" standards of presentation and language.
The contrary argument that always gets trotted out is that this prevents "new" or radical ideas as they tend to be suppressed (overtly or covertly) by "established" ideas. While I think this is a danger, its true in any system that contains "vested interests" or "assumptions". Revolutionary change (great strides) is difficult for any system that is geared toward evolutionary (many small steps) change. This is more a factor of the mechanism of the system than the desire or intent of any one person within the system.
So what about algorithmic quality filters? Do they allow or reject revolutionary content? My guess would be ... it depends! An algorithm is a reflection of the programer(s) who created it. This usually means that it carries all their assumptions and flaws. More simply put, is the algorithm essentially a black list or white list model. Algorithms that are smart enough to do semantic analysis are just not yet feasible on the large scale, but hopefully they will be making an entrance soon and go some way to solving this type of problem.
Wednesday, April 27, 2011
Fun with matching algorithms
http://www.theregister.co.uk/2011/04/26/amazon_pricing_algorithms/
This is a fun little example of matching algorithms with limited inputs and a very short feedback loop. What do you get.... uh, growth... duh.
This is a fun little example of matching algorithms with limited inputs and a very short feedback loop. What do you get.... uh, growth... duh.
Friday, March 25, 2011
Dueling Algorithms Paper
http://web.mit.edu/newsoffice/2011/dueling-algorithms-0318.html
This is interesting. Must read the paper.
This is interesting. Must read the paper.
Wednesday, February 23, 2011
Friday, January 21, 2011
DNA, Game Theory and Risk in partner selection....
http://blog.okcupid.com/index.php/the-mathematics-of-beauty/
This is quite thought provoking. There are a slew of confounds and different directions to take the research... but that's not what interests me initially.
I was interested in the two types of graph patterns that were being discussed. Not if they were correct, just the thoughts that I had trying to explain them.
Firstly, the two profiles are essentially the "Normally Attractive" and the "Split Hot or Hate" shapes. It got me thinking about DNA and partner selection along with some game theory.
The first point is how to explain the "Split Hot or Hate" graph.
If we assume that the "Normally Attractive" pattern represents a female partner who has effective enough genes to be successful in the current environment. This means that her DNA has been selected for and reinforced across a significant part of the population enough to be considered the "normal" look. (This also assumes that appearance and whatever else was being overtly or covertly evaluated by the fairly simple measure is tied directly to DNA or social DNA... anyway)
So people who are outside this "normal" are aberrant in some way. This may be because they express non-normal characteristics. Now if the environment is static, then the conservative strategy for picking a mate would be to always pick a "normal" mate and work on the assumption that others have taken all the risks and normal is good enough to survive. The higher risk strategy is to look for someone who exceeds normal in some fashion and try to breed with an exceptional partner.
However in the event that the environment is not stable, picking a "normal" is no longer the risk free or conservative strategy. Because "normals" are adapted for the environment that was previously stable and may not be so tomorrow. (The question is if our DNA is smart enough to express that kind of strategy via testosterone or just uses blind selection to deal with this scenario.... which is simpler do you think?)
So how does this explain the "Split hot or hate" pattern? I would suggest that risk taking in mate selection is essential for a healthy population and the ability to explore different mutations. But its probably only valuable if a small-ish part of the population engages in these risky ventures. The rest of the population should be steered toward only breeding with "normal" partners and avoiding anyone "abnormal". This way the population should have a large central pool of "normal" stock with a smaller pool of mutants ready to take advantage of any useful mutations or environment changes. So these graphs represent the difference between "normals" and "abnormals". While the "normals" get a response pattern that suggests they are acceptable to conservative and risk taking partners, the "abnormals" are either highly attractive to risk taking partners or highly repulsive to conservative partners.
This study is possibly more interesting because it was done on the male response to female potential mates. The dis-equality between the male and female illustrates just how wildly different mate selection is on the two sides of the fence.
The cost to a male of breeding is relatively tiny if there is no husbandry involved. While the cost to a female is potentially a year or so of her life or ~ 1/15 - ~1/20th of her possible breeding events(Assuming starting breeding about 15 and being fertile until mid to late 30s). So for Males to breed with a female outside the "normal" pool is fairly low cost/low risk. While for females its a much higher cost, but with the same risk.
Once you involve husbandry and the social conventions of mating, it gets much more complex but the underlying biology is the same.
I would predict that a similar study from the female point of view would show a similar pattern for the "normal" pattern males but a much smaller percentage of "hot" ( high risk but worth it) types and a much larger percentage of clear "hate" group.
This reflects the percentage of "conservative" choices being higher for women due to the cost and risk of breeding with an "abnormal" being higher.
It would be interesting to break the scoring given to each partner down into a set of finer grained plus or minus scores. IE.
Like their profile information (score 1-5)
Like their picture (Score 1-5)
Overall like ( Score 1-5)
Later....
This is quite thought provoking. There are a slew of confounds and different directions to take the research... but that's not what interests me initially.
I was interested in the two types of graph patterns that were being discussed. Not if they were correct, just the thoughts that I had trying to explain them.
Firstly, the two profiles are essentially the "Normally Attractive" and the "Split Hot or Hate" shapes. It got me thinking about DNA and partner selection along with some game theory.
The first point is how to explain the "Split Hot or Hate" graph.
If we assume that the "Normally Attractive" pattern represents a female partner who has effective enough genes to be successful in the current environment. This means that her DNA has been selected for and reinforced across a significant part of the population enough to be considered the "normal" look. (This also assumes that appearance and whatever else was being overtly or covertly evaluated by the fairly simple measure is tied directly to DNA or social DNA... anyway)
So people who are outside this "normal" are aberrant in some way. This may be because they express non-normal characteristics. Now if the environment is static, then the conservative strategy for picking a mate would be to always pick a "normal" mate and work on the assumption that others have taken all the risks and normal is good enough to survive. The higher risk strategy is to look for someone who exceeds normal in some fashion and try to breed with an exceptional partner.
However in the event that the environment is not stable, picking a "normal" is no longer the risk free or conservative strategy. Because "normals" are adapted for the environment that was previously stable and may not be so tomorrow. (The question is if our DNA is smart enough to express that kind of strategy via testosterone or just uses blind selection to deal with this scenario.... which is simpler do you think?)
So how does this explain the "Split hot or hate" pattern? I would suggest that risk taking in mate selection is essential for a healthy population and the ability to explore different mutations. But its probably only valuable if a small-ish part of the population engages in these risky ventures. The rest of the population should be steered toward only breeding with "normal" partners and avoiding anyone "abnormal". This way the population should have a large central pool of "normal" stock with a smaller pool of mutants ready to take advantage of any useful mutations or environment changes. So these graphs represent the difference between "normals" and "abnormals". While the "normals" get a response pattern that suggests they are acceptable to conservative and risk taking partners, the "abnormals" are either highly attractive to risk taking partners or highly repulsive to conservative partners.
This study is possibly more interesting because it was done on the male response to female potential mates. The dis-equality between the male and female illustrates just how wildly different mate selection is on the two sides of the fence.
The cost to a male of breeding is relatively tiny if there is no husbandry involved. While the cost to a female is potentially a year or so of her life or ~ 1/15 - ~1/20th of her possible breeding events(Assuming starting breeding about 15 and being fertile until mid to late 30s). So for Males to breed with a female outside the "normal" pool is fairly low cost/low risk. While for females its a much higher cost, but with the same risk.
Once you involve husbandry and the social conventions of mating, it gets much more complex but the underlying biology is the same.
I would predict that a similar study from the female point of view would show a similar pattern for the "normal" pattern males but a much smaller percentage of "hot" ( high risk but worth it) types and a much larger percentage of clear "hate" group.
This reflects the percentage of "conservative" choices being higher for women due to the cost and risk of breeding with an "abnormal" being higher.
It would be interesting to break the scoring given to each partner down into a set of finer grained plus or minus scores. IE.
Like their profile information (score 1-5)
Like their picture (Score 1-5)
Overall like ( Score 1-5)
Later....
Thursday, January 20, 2011
Saturday, May 29, 2010
Search strategies
Ever lost something in your house? Thought you knew where it was but turns out you didn't? When you go looking for it, its just not there. What do you do?
Search nearby? Search in ever widening circles around the spot where it should be? Try to retrace steps? Look in the lost-and-found basket? Ask someone else? Systematically begin searching everywhere? Quarter the house and start a search grid? Do a sampled search of specific areas? Try to apply probability to where it most likely could be? Employ search agents( not your children... really, it doesn't work.)
There are some interesting strategies for searching for an thing in an unknown environment. There are a few ways to try to optimize the search but they are often dependent on properties of either the thing, the environment or the search tool(s). Not always generalizable.
As you might have guessed, I have lost something.
Search nearby? Search in ever widening circles around the spot where it should be? Try to retrace steps? Look in the lost-and-found basket? Ask someone else? Systematically begin searching everywhere? Quarter the house and start a search grid? Do a sampled search of specific areas? Try to apply probability to where it most likely could be? Employ search agents( not your children... really, it doesn't work.)
There are some interesting strategies for searching for an thing in an unknown environment. There are a few ways to try to optimize the search but they are often dependent on properties of either the thing, the environment or the search tool(s). Not always generalizable.
As you might have guessed, I have lost something.
Monday, April 26, 2010
Archive from 5_6_07 - Knowledge harvesting
Just had a thought about harvesting knowledge in a public space.
Articles on web sites are written and posted by the author. In general there is no structure or expectation for feedback or update. They are "published". The only knowledge captured is that of the author. A hierarchy of one.
Compare that with a site like codeproject (www.codeproject.com)where an article( specifically about some programming topic ) is written and posted by the author but has a threaded forum attached to the bottom where comments and discussion can be collected and retained. The author remains the top level arbiter of the discussion with ( I assume ) edit rights to both the article and the discussion. The discussion participants retain edit rights to their contribution. A two level hierarchy.
Look now at something like slashdot. (www.slashdot.org) Here the author of the post contributes it then its chewed upon by the community. The signal to noise ratio is so low in the following discussion that the author has little hope of managing or replying to the discussion posts. The site employs a voting system to attempt to help the signal to be elevated above the noise. The author looses edit rights to the article; which are taken over by one of the site editors. Discussion posts are editable by their contributors and there is influence by moderation and voting. This creates a complected three level hierarchy with some complicated and non-deterministic rules for predicting what the final knowledge collection might look like. This in itself is an interesting example of a system to harvest knowledge out of an essentially chaotic community. The beauty of the system is that it works. The drawback is that the voting system can be 'gamed' and will corrupt the process if it happens on a large scale.
One of the key elements is that the dynamic updates promote the signal upward while 'modding' down the rubbish. It is not discarded but is visually diminished and tagged so it can be filtered out if desired. However at any time later, this rubbish can still be examined by researchers.
Compare this to a wiki system. Again it uses an article as the cornerstone of the knowledge accumulation. This article then is modified directly by random contributors. They can add, edit or delete information. Most wiki's seem to have the capacity for a simple comments system which is attached to the page. This is un-threaded and in my experience relatively unused. However it does make for an ideal place to ask questions and make requests. The problem being that without a responsible maintainer, there is no one to service those requests reliably. Its the responsibility of the 'community' to chaotically make improvements.
Wiki systems have an interesting 'rollback' feature that allows changes to be 'undone'. This mitigates the damage of authors work being deleted maliciously. However as the author and the subsequent contributors are effectively peers, who is to say that the original author's contribution is better than the deletion made by the peer.
Wiki systems spawn pages. These are hyper linked to older pages and the resulting knowledge base becomes a standard hyper linked mess. There is no way to automatically assess quality or topic or relevance to any particular subject different from a normal keyword search or page rank system.
Now look at discussion forums. Commonly called 'forums'. These usually have a threaded and date sorted post structure. A number of separate 'forums' are often collected under a major subject heading or site. These sub-forums are usually just arbitrary separations of the topic area or activity of the 'community' that they are servicing. The membership of the community is often fairly chaotic but share some commonality. Either a topical interest or some common property.
A community member can make a 'post' at the root level of the forum. This post can then be commented upon by other forum members who add to the thread. The original author has edit control over the post and any subsequent comments they might make in the resulting thread. These threads can branch into sub thread; making for a hierarchy of comment and response.
The root of the forum has three classes of post. There are 'recent' posts that are sorted to the top of the list by date. 'older' posts that have moved down the list simply due to the date they were posted and what are called 'stickies'. These 'stickies' are often posts addressing common topics or rules of conduct for the forum. They are fixed at the top of the forum and remain there irrespective of the date sorting.
The only other meta data that can be automatically derived from a post-thread object is its activity, number of posts, frequency of posts, statistics on who has posted, size of the posts and the number of thread branches. Otherwise the knowledge encoded in the structure will slowly drift away from the root of the forum.
Should write a paper on this at some point....
Articles on web sites are written and posted by the author. In general there is no structure or expectation for feedback or update. They are "published". The only knowledge captured is that of the author. A hierarchy of one.
Compare that with a site like codeproject (www.codeproject.com)where an article( specifically about some programming topic ) is written and posted by the author but has a threaded forum attached to the bottom where comments and discussion can be collected and retained. The author remains the top level arbiter of the discussion with ( I assume ) edit rights to both the article and the discussion. The discussion participants retain edit rights to their contribution. A two level hierarchy.
Look now at something like slashdot. (www.slashdot.org) Here the author of the post contributes it then its chewed upon by the community. The signal to noise ratio is so low in the following discussion that the author has little hope of managing or replying to the discussion posts. The site employs a voting system to attempt to help the signal to be elevated above the noise. The author looses edit rights to the article; which are taken over by one of the site editors. Discussion posts are editable by their contributors and there is influence by moderation and voting. This creates a complected three level hierarchy with some complicated and non-deterministic rules for predicting what the final knowledge collection might look like. This in itself is an interesting example of a system to harvest knowledge out of an essentially chaotic community. The beauty of the system is that it works. The drawback is that the voting system can be 'gamed' and will corrupt the process if it happens on a large scale.
One of the key elements is that the dynamic updates promote the signal upward while 'modding' down the rubbish. It is not discarded but is visually diminished and tagged so it can be filtered out if desired. However at any time later, this rubbish can still be examined by researchers.
Compare this to a wiki system. Again it uses an article as the cornerstone of the knowledge accumulation. This article then is modified directly by random contributors. They can add, edit or delete information. Most wiki's seem to have the capacity for a simple comments system which is attached to the page. This is un-threaded and in my experience relatively unused. However it does make for an ideal place to ask questions and make requests. The problem being that without a responsible maintainer, there is no one to service those requests reliably. Its the responsibility of the 'community' to chaotically make improvements.
Wiki systems have an interesting 'rollback' feature that allows changes to be 'undone'. This mitigates the damage of authors work being deleted maliciously. However as the author and the subsequent contributors are effectively peers, who is to say that the original author's contribution is better than the deletion made by the peer.
Wiki systems spawn pages. These are hyper linked to older pages and the resulting knowledge base becomes a standard hyper linked mess. There is no way to automatically assess quality or topic or relevance to any particular subject different from a normal keyword search or page rank system.
Now look at discussion forums. Commonly called 'forums'. These usually have a threaded and date sorted post structure. A number of separate 'forums' are often collected under a major subject heading or site. These sub-forums are usually just arbitrary separations of the topic area or activity of the 'community' that they are servicing. The membership of the community is often fairly chaotic but share some commonality. Either a topical interest or some common property.
A community member can make a 'post' at the root level of the forum. This post can then be commented upon by other forum members who add to the thread. The original author has edit control over the post and any subsequent comments they might make in the resulting thread. These threads can branch into sub thread; making for a hierarchy of comment and response.
The root of the forum has three classes of post. There are 'recent' posts that are sorted to the top of the list by date. 'older' posts that have moved down the list simply due to the date they were posted and what are called 'stickies'. These 'stickies' are often posts addressing common topics or rules of conduct for the forum. They are fixed at the top of the forum and remain there irrespective of the date sorting.
The only other meta data that can be automatically derived from a post-thread object is its activity, number of posts, frequency of posts, statistics on who has posted, size of the posts and the number of thread branches. Otherwise the knowledge encoded in the structure will slowly drift away from the root of the forum.
Should write a paper on this at some point....
Subscribe to:
Posts (Atom)