Wednesday, November 17, 2010

Natural Language Processing

Good resource both on Haskell and NLP.  Nothing that can't be done in another language. I feel much more comfortable mapping this problem space to C++ than to Haskell, simply because I'm more fluent. I also have a sneaking suspicion that the myths about Functional Programming are less about any intrinsic properties of the language and more about the person holding the hammer....

I have actually done some of this in VBA, which if anyone is interested, is painful due to the ugly data structures and having to build everything yourself. I know I can use .NET containers etc but its still ugly because its not my favorite hammer and its dog slow, hangs unpredictably and ... well its just dog slow(very slow dog... ignore obvious edge cases in this metaphor) I can make any tool work given enough time and energy but some just make the job harder in ways that the word "harder" does not express completely or gracefully. Think of VBA as a plastic toy hammer with an asthmatic squeaker .... C++ on the other hand is a bit like an antique 2T power hammer with a loose linkage in the return spring and a lumpy anvil but damn it can hit the problem hard....

Imposter Syndrome

I like this post. Its well constructed, short and punchy and speaks on a topic that resonates with me (and some other people I can think of). In the contextual abstract its a beautiful post. (Although the first post in the comments is a total fanboy tag) anyhoo... as for the actual content, its slightly disturbing. I would suggest that while I identify with the Imposter Syndrome, in reality I would probably conclude that not only have I been there but I have taken the escape hatch route. I did not go to the happy place, I have gone to the safe place.

This line of thought becomes a complicated tangle of self doubt, supposed objective analysis, excuses, rationalisations and unfulfilled dreams until reality crashes in, gets dismissed as excuses, exits stage left in a huff and proceeds to play devils advocate from the wings dressed in the guise of a "Gandalf-ian" old wise man getup. 

Am I confusing the pressure of unreasonable expectations and workload with the fear of failure/discovery issues that were discussed in the post? Perhaps

Am I pushing myself to learn new things or am I cruising at a safe altitude? Nope. New things every damn day.

Am I building something bigger than myself? Nope. Its just a job that will exist long after I have gone.

Am I endlessly passionate about what I am doing? Bits of it. A great deal is politics and ephemeral bullshit... but I find value in it all. It stretches me in other ways that are not always comfortable or pleasant... so I would phrase it as: "Its a challenge every day".

On the topic of being in a safe place vs on the scary bleeding edge... I think as a parent and provider for children, the safety and security aspects trump the bleeding edge thrill stuff.  Its irresponsible to expose your children to risk and stress. Full stop. There is no "but..." arguments after this point. I can think up any number of rationalisations and guilt trips about how I'm wasting my talent etc but its not about me and I accept that I may look like a cop-out, or I'm hiding behind my children so I don't have to play with the big boys... but providing a safe and low stress environment for the children comes first, second and third otherwise I'm a failure as a parent. So while every guilt trip still lands and hurts, I suck it up and do what I know is the right thing. I find ways to enjoy a job that is not thrilling every minute of the day, I humbly accept pay that is far from the bracket that I was aiming at, I disappoint the dreams of various people who are looking for "great things" from me ( or great money...), I trade all this for a reliable pay packet, a quiet neighborhood with trees and dogs and schools and beaches where my children can grow up and learn without having to worry about all the other stuff.

Do I still dream? Is the hungry urge to beat the world at something still there? Do I still want to do amazing things and create fantastic tools that I can just about see how to do? Hell yessssss! But I take care of business first.  Walk carefully through the minefield. Make safe choices, reduce risk, manage money, balance the budget, keep the cart on the tracks, put one foot after another, ignore temptation, be the money cop, ignore opportunities, stay focused, play the long game, avoid regular fixed costs, reduce debit, stay with what you know, save for a rainy day, don't explore shady service providers(phone companies, banks, internet providers etc),  be the dull dependable safe sunscreen wearing boring person that puts food on the table, a roof over their heads and endless stimulation in their minds.

Oh and don't ride motorbikes. I miss my bike every day. Its was 300kg of big, black crazy risk taking behavior. It reminds me not only of what I have given up but inversly what I have given it up for. Better than a tattoo because the pain does not fade. (even for a huge battle cruiser, it could hit 180kmh at red line in 5th... so I hear)

Now I have the thrill of debugging spreadsheets and labeling equipment.... no comparison really.

Tuesday, November 16, 2010

Tenacious C IDE

This looks like an interesting product. Need to investigate more at a later date.

RESTful services model

This is an interesting article on the architecture of a RESTful service. I've been running into this term for a while now and have not really investigated it before. So this was a very useful read. Funny now that I've pasted the link and looked at where the article was actually hosted it make sense why it was so polished.  Shows how little I'm actually reading page headers and such....

Curating a blog

Every so often I have the need to go back and clean up old blog posts. It's, strangely enough about this time of year, most years. The research students are done and its time to clean up, round up the assets, restock the supplies in the labs, archive the tools and software, catch my thoughts, and generally look at the year in review. But back to the point....

Curating a blog. I guess I'm not having any new ideas that have not been had by others who maintained diaries or any other sort of longitudinal writing. It becomes a body of work and patterns start to emerge.
I wonder if by cleaning it up and curating it I'm actually destroying some of the historical issues that would otherwise have been interesting to me in the future. Bad spelling which was a result of starting with a blog package without spell checking (and being too lazy to manually copy/paste into a spell checker). This in itself is valuable insight into why I was blogging at the time. It was about cathartic train of though writing. Just hacking something onto the page without too much thought about why or who or what. That is being lost a little at a time as I go back over old writing, see it with current eyes and standards and "clean it up" to suit my current frame of reference. Is this a good thing?

Does anyone care or will they ever care?

In this respect paper has more authenticity. I certainly have some piles of paper with random writing on it and directories on old computers full of random writing. However its less structured. There is little sense of a timeline because so much of it has been moved around from backups and disks that its lost its original time stamps, or its actually been re-edited at some point. The other side of the coin is that the paper writing is also dateless so its not that different I guess. Mostly just random snippets of bits and pieces. Every so often I try to find the time and energy to digitize it and extract all the supposed pearls but like all piece of string projects, the value is pretty trivial compared to the work to transcribe it all.
Its not like I actually believe there are any great insights lost in it. Mostly its just childish fragments and scenarios that were meaningful to me at the time.
Midlife crises suck.

The boy is starting to get quite verbal now. He's putting good meaningful sentences together and its possible to get a bit of a conversation going. Two or three sentences anyway but he's expressing himself a little more than just frustrated screams or inarticulate noises. Every day its a bit clearer.

The girl and I have successfully built our second robot together. This one is a OWI 5DOF arm with a USB interface. It's from a kit so its not as big a challenge for her. I wanted something that would work once we has put it together rather than a home brew system that would have taken endless tweaking and frustrating debugging. She still only has an attention span of a couple of minutes at a time so we need fast results and a predictable outcome. Its been a great little project so far.
The firs robot we built was about 6 months ago. It was another kit system using Capsela components for a simple walking robot with pose-able arms. Very trivial but a good first project. We had results in about 10 minutes without any tools. Like most Capsela systems its not featured on any websites and seems to have never existed. I found it in a little toyshop an it seemed to be the only one they had.  Anyway....
Now I have to find the next challenge.

Wednesday, November 10, 2010

Character differences between WRPG's and JRPG's

Very insightful article about different styles of character and how that ties back to the social models the games derive from.
Lots to think about.

Tuesday, November 9, 2010

Supervisor pattern

A heavy post on cloud patterns. Need to re-read this and have a think about it.

Improving the performance of a Shared Access Database

I've implemented the trick of holding the lock file open and it seems to be helping. Counter intuitive to all the energy I have spent releasing resources in other languages and systems but anyway.... it works. I am also doing a lot better with caching results and doing my joins in code rather than via SQL. Anyway, back to the bug hunting...

Monitor Arms

Finally found a source of my favorite monitor arms in small quantities.  Scorptec have them under the brand name "Manhatten". They are cheap, solid and black and work brilliantly. I have about half a dozen already from a different supplier under a different name. Not shiny but they just work.

Ergonomics Information

Good post on a variety of issues for programmers and some reasonable solutions.

Learn Python tuts

Something for my endless supply of free time.

Bit hacks to remember

These are some useful techniques to keep fresh for the upcoming embedded programming.

40 hour week

This is an interesting post about working within a bounded time frame. I have had a week or two to think about the issues raised and I still keep coming up with new and valuable insights from it. Its worth another read at some point.

Firesheep plugin

This is a little old but still funny. I'm interested in this from a security point of view but also from seeing what the reaction to it is. Its one of those things that could catch fire and take off or disappear into the background noise. Its an interesting experiment on public scrutiny and security by obscurity.  The techniques to exploit this hole have been around since cookies were invented and abused for session management over insecure networks... so far its passed about 1.4 million search results on google using "firesheep -sheep". The top ten pages are all 100% articles on Firesheep so I figure the rest of the results are probably pretty good.  LOL.  Way to shine a light on the issue. Lets see if anything happens.

The general reaction has been two fold. Firstly all the tech press is generally cheering the political objectives while recommending countermeasures. Secondly the hysterical non technical press is decrying the existance of such a terrible weapon... blah blah blah.

Another interesting aspect is the ecosystem of countermeasure tools that are popping up. BlackSheep and FireShepard are the two that have sprung fully formed to offer a solution for the ignorant. Does that not strike you as suspicious?  I have read that BlackSheep is actually a DDOS attack client which I find much more credible than that it magically has some capacity to reach out and touch a passive sniffer application. The description of how it works is kinda credible but not if you know much about DNS and how FireSheep actually works. Even if its exploiting a weakness in FireSheep, its not actually dealing with the underlying issue that is being highlighted. It would be trivial to rework FireSheep to be impervious to BlackSheep's supposed technique.

As for FireShepard:

This page has a lightweight description of how it claims to work. Again its basically trying to attack a weakness in the Firesheep tool rather than patch the problem that FireSheep is highlighting.  Also FireShepard would probably breach the terms of service of any reasonable network because it works by intermittently flooding the network with rubbish packets. This sort of activity would probably set off all sorts of DOS attack detectors, Intrusion systems and just generally piss off any network admins who caught you using it.  Its the equivalent of turning on the sprinkler system in a whole building to put out a single candle (that may or may not be there). And just consider the chaos if one paranoid user on the network starts talking about it to their co-workers and encourages them to also install it.  You then have multiple people intermittently DOS'ing the network segment. Genius.... (Sarcasm)
The first tool sounds like its a tiny step from being outright scamware if its not already malware. The second sounds like a poorly thought out tool with marginal hope of fixing the problem but much larger potential for getting the user banned or prosecuted.

Nothing has turned up about dealing with false positives or the social consequences of detecting an attacker and how to deal with it ethically or safely has shown up yet.  I would assume that the common witch hunt rules would apply. If you think someone is running a sniffer on the network, you can unilaterally employ the "strike first" approach and burn them publicly so you feel all safe again.  Since there is no actual evidence (unless your facebook profile has been hijacked by a completely incompetent person who signs all their fake posts with their real name... but then how would you even prove that that was their real name?  Endless fun with digital forensics.

So we have a scary mix of paranoia, uncertainty, ignorance, exploitative tool developers, no useful solutions from most of the affected sites and a bubbling pool of anger, distrust and the usual illusion of invulnerability that internet users get when they feel safe and anonymous. Nothing bad could happen here...

Stats with Cats Blog 10 fatal flaws

I should hand this out to all the research students as they ask the same questions again and again....

Irony Language Implementer

This looks useful... but not right now. Could be used for implementing domain specific languages that leverage the .NET runtime. May be a good solution to create an embedded language for inclusion in experiment generations tools.

i fix it site

This is an interesting site to keep an eye on. Not particularly rich with detail yet but growing. They need some better in depth diagnostic information for many of the devices rather than just a list of which screws to remove. But we live in hope.

Monday, November 8, 2010

Narrative structure for RPG

I am back to thinking about suitable narrative structures for CRPG games.

My current breakdown is something along the lines of:

* Linear narrative - Essentially there is one start and one ending, there may be more or less variability in the middle.  (Think Max Payne )

* Broom narrative - Similar to the above but with the addition of a number of different endings. (Stalker is an example ) 
* Serial missions - One mission at a time usually with a strong theme. Can do branching but very expensive to develop whole mission modules that may not be played through.
* Mission Loom - From a single start point the rest of the game is composed of many small missions that overlap and weave together. Good for pickup games or for the OC gamer who wants to "complete" everything in the walkthroughs.
* Mission Loom with main thread(s) - One or more primary missions are presented to try to tie all the mini-missions together into a common theme? (Think Fallout 1,2,3 etc)

These are essentially all fragile systems that have modular segments that join together to create the narrative context for the play activities. They are fragile for a number of reasons, the first being that they have no capacity to be resilient to any errors in the chain. If one module fails ( for whatever reason) it conceptually leaves a gap. Even if the player somehow crosses the gap and gets back into the game, they have experienced the cognitive dissonance of having to "exit" the flow of the game to deal with the broken game engine/level script/ whatever bug.  Obviously this is undesirable.

The second issue of fragility is to do with the Users perceptions of whats going on. Its fine to be able to tune a level until most users get it most of the time, but for complex storytelling that involves any subtlety ( which the debate is still open as to how many people would actually buy a game with too much subtle storytelling...stuff but anyway) there is no way to gauge if the user is following the storyline at a level they want or care about. (I use care to represent their level of engagement in the storyline rather than just their engagement in the game activities which can be a different kind of bear tickling)

So, to recap, the player can loose interest, loose the thread or fail to engage with it enough to differentiate the meta story from the game activities. (Not see the forest for the trees... so to speak)

The other sort of fragility I was thinking about is the temporal aspect. How to deal with the amount of playtime that the player wants to commit vs the time required to get through the narrative. Can the player commit more time if they enjoy the story or does the story have to fit the amount of time the player wants to commit?  Both of these scenarios have ugly issues associated.  Players are getting a little tired of "sticky" games that demand attention. (This whole issue has been soured by the MMORPGs and their subscription models.)
So players want a quality experience not a skinner box.  Something that enriches their life rather than sucks it dry.

This is a similar issue to the one faced by television serials. Keep a narrative arc or make each episode self contained. Keep them wanting more or give them what they want? The eternal dilemma for media producers.

I have been reading an interesting post by someone about working inside a bounded time frame that has some interesting dimensions for writing. These ideas have mixed with an analysis I recently read of the "Robert Jordan - Wheel of Time" fiction series.  One point of view is about how forcing us to work within a clearly bounded time frame helps sharpen everything you are doing while on the other hand, the book series was criticized because it wandered at times without boundary and so the quality suffered.

The point being that forcing a scenario or a narrative into a fixed time frame may be no bad thing. Especially a reasonably short one. Cinematic scripts are usually 120 pages which forces them to keep it tight (conceptually. Lets not argue about all the exceptions to this rule that exist.)

So how would that be applicable to game narratives, assuming game narratives are even well enough defined to be called such a thing.
If we are looking at a simple linear narrative, its easy to apply a fixed time to the narrative. It just keeps on ticking no matter what the player does. This forces the player to get with the program and stay with it. In effect punishing them pretty severely for making any mistakes or wandering around exploring.

Pick this up later.

Wednesday, November 3, 2010

Swapping keyboards is complicated

My current favorite typing tutor. I have swapped to a Kinesis Freestyle keyboard and its been a bit of an adaption curve. This site provides some useful exercises to help get me back up to speed.

I started by moving the whole keyboard to the ideal 30 degree split but found it too much too soon and put it back together. I have since been progressively moving the angle and adapting slowly.  I am still not a fan of the key layout. I appreciate the loss of the numeric keypad but I have some kind of brain failure with the backspace and delete keys. I constantly seem to be getting them backward. Mainly because the delete key is approximately where I expect the backspace key to be which works the reverse of my intuitive expectation, then when I realize the mistake, I have to consciously think about the solution and it gets very messy. Suffice to say its working itself out but breaking long habits is challenging without the regular practice of a keyboard trainer.

The other irritating thing is the keys on the left hand side of the keyboard. They are essentially useless. They are sort of shortcut keys to reduce RSI but need to be completely learned. My biggest problem is I keep trying to feel for the lower left ctrl key which used to be the left most bottom key, but now its not. I keep hitting the shortcut key for context menus instead.  Like I said, irritating but not life ending.

I have also been using the Evoluent upright mouse for a while. I would say that my adaption to it has been less useful. I still habitually try to hold it from the top down. And I find my coordination and precision is pretty crap when I do hold it the right way. Rather than driving from the wrist, I find I end up moving from the elbow which is much less precise. Probably from lack of practice. Still its a whole new set of muscles and nerves to train so it would be good to get some mouse training games happening.
I happened to do some timed testing tasks with it the other day and my coordination was a bit slower than usual and it made me feel like I was fighting the mouse rather than it being an intuitive extension of my hand.

Blah.  I am still not game to try the Kinesis advantage keyboard. I'm worried that if I adapt to that I will get even less capable with all the normal keyboards I use every day. As a Tech I need to be able to work on any of the computers in the place and not being able to drive the keyboards I find would be a big disadvantage. I'm still wondering if I can maintain two similar sets of competency without making a mess of both (as usually happens for me)

VOIP SDK for Telemedicine app

This looks like a useful product for some of the telemedicine projects that are floating around. It uses SIP for brokering so I can pretend like I know something about it from my past research. Good to see someone making it more accessible. Version 2 will be very interesting.

Brain Science and Brain Training

Need to keep track of this as a brilliant set of samples. Kinda irritating because I have had to hack up a bunch of these tests for various experiments in the labs.  These ones are much prettier than mine which suggests I need to get back to some flash at some point. Probably just before it all dies from HTML5... figures.

Fastflow Parallel Programming framework

This looks very interesting. When I suddenly come across a bottle of "free Time" I might just have a drink and a think...

Clang and LLVM for C++ refactoring

Thank the Assembler, I don't have to do it myself. Not that I'm delusional enough to think that I could have, rather that I could see the need but there appeared to be no solution other than me doing it (somehow) myself.

Finally, I can start to dream of getting some re-factoring tools working for C++ and after that we can talk about high level transformation of code... better static analysis.... design extraction.... whooot!  Well a boy can dream can't he?

Monday, November 1, 2010

Philosophy of hacking

Found an interesting post on Lifehacker

It wraps a philosophical context around the activity of hacking and the mindset. I doubt that many hackers have ever conceptualized their motivations in such a way but the piece does have a certain resonance.  I certainly have never felt that disobedience was my motivation. Usually it was just my frugal farmboy upbringing that motivated me to fix whats broken and extend the life of anything that is useful. Growing up my reality was that if I couldn't fix it or make it, I didn't have it.

But back to the article. I can see the association that the author is making, and I think it has merit. Mainly because I can't really think of a good argument against it, but I am still wary of suggesting this argument captures the general motivation of most or all "hackers".

The argument gets a little shaky when you drill down into the semantics. "Disobedience" semantically is not really the over arching motivation for most hacking projects, I would suggest that in the way the author was using the term, its more a means to an end. In that a hacker has to "disobey" the rules not to do what they are doing, but they are doing it for "X" reason. "X" may be anything from fixing a bug, to adding features, to working out how something works to trying to impress someone and get lucky ... whatever. Disobedience is simply a means to an end.

The interesting implication that the author did not make was that the willingness and the ability to be disobedient are essential to successful hacking.  The idea of not being afraid to "void your warranty" and not being afraid of "bricking it". This is essentially risk taking behavior in another form along with a degree of confidence combined with technical experience that leads one to suspect that something is possible.

To extrapolate this train of though, its possible to suggest that the environment of warranties and license agreements and all manner of other "rules" sets that impede the fairly natural instinct of "taking it apart to see what makes it tick" is an environment that habituates curious people to ignore these rules and thus be less worried about them in other aspects of their lives. Life is just more straight forward when you ignore most of these unenforceable rules ("Thou shalt not mod thy game console") and threats ( "void your warranty") and get on and hack your device/software/web service/file format etc into whatever shape you currently need.

I would also suggest that hacking is an emergent phenomenon in any complex structure. Once a system gets sufficiently complex, opportunities emerge for something else to find more direct ways to exploit resources within that system. This may not be to completely break the system, rather just to subvert parts. Look at all the parasites inside ant and bee colonies. These genomes have accidentally found strategies to hack an otherwise all encompassing system with no "legal" opportunities.  The result being that they make out like bandits.

Alternately, look at any large organisation and consider all the "proper channels" and the way people naturally find ways to get their jobs done even when parts of the system are malfunctioning ( bad manager, poor communications, horrible enterprise software etc) people find ways around things and the more they get used to going around the obstacles, the less they "follow" correct procedure in other issues. Eventually they habitually look for the most straight forward way to achieve the ends and the system re-aligns or sections are discarded. 

Hacking is all around us. Every day I see people who have no idea what's inside a computer or what to do with a hex editor hacking away at corporate systems. They use telephones to talk to other parts of the organisation and avoid sending emails through "proper channels" or they feed systems information and data that they know will be accepted, even though its technically wrong, just to avoid having to deal with broken enterprise software systems. They talk their way around problems and obstacles to get things done. These are not the idealized "geeks" taking apart technology and making things in their bedrooms but its still hacking systems to get results that are otherwise not possible. Are these people "disobedient"? In a way, certainly. Is their motivation "disobedience"? I think not. In many cases its very much the urge to keep the system working and to fit in and not be the "exception" that forces them to hack the system.

This leads to an interesting question. Will they continue to hack the system when the need is no longer there. Does this "breed" disobedience or will these people return to being rule abiding citizens once the enterprise software is fixed or the passive aggressive supervisor has been fired? Is "hacking" simply a case of people "finding the path of least resistance"?

I think there are some people who, having had to learn the skills and techniques of hacking their particular problem will never forget these skills. The question is if they will ever again be in an opportunity to need to employ them. For other people, I would guess that the discomfort threshold required to have to hack a system is sufficiently high that they will probably never go back to it, once the itch is scratched. Because, at the end of the day, Hacking is hard, risky and often involves swimming against the flow in many ways. This can include threats to comfort, safety and security ( loose a job, legal action, fines, social stigma etc) so these put pressure on some pretty basic motivations. Which means that generally the motivation to hack is dampened by many of the rules that the "system" has been constructed from, otherwise it would not be a system in the first place.

Ok, enough D&M.