Pointless philosophical debate - Disengagment, Debugging and Futility

I listened to an Honours presentation a few years ago called:  "Knowing When To Quit: Self-Focused Attention Enables Disengagement, Positive Affect Predicts Reengagement Tendencies. (Tristan Hazelwood)".

It was on one of those aspects of psychology that I had never considered before.  That of "task disengagement"... or the ability to quit doing something. I found this quite personally intriguing becasue I had always seen "quitting" as a failure state and even though I have quit a lot of things in my life... it was always with this sense of personal failure.

Tristans experiment looked at how long people will continue to attend to a "futile" task before stopping doing it.  There were some manipulations as well... which are not relevant.

I had never really asked myself this question or the extension questions.

How long do I keep doing futile tasks? 
How do I know a task is futile?
What if a task seems futile but is ultimatly not?
What if the effort required to complete the task is disproportionate in comparison to the reward?

Anyway,  this gets back to my today issue:  Fixing bugs in other peoples code. 

I have just spent way too long trying to debug why the wrong favicon was being displayed in a particular browser (Firefox).  This required me to:

• Verify my assumptions about the local copy of the site files (Still broken)
• Verify my assumptions about the server copy of the site files (Still broken)
• Reload everything in Firefox
• Test in multiple other browsers
• Read up on how Firefox handles FavIcons
• Break into the sqllite db used by Firefox to cache the favicons
• Clear the cache, reload, test, clear again, clear a different way... test-fiddle cycle until I got the combination correct and finally expunged the old favicon from the chain of server-browser-cache rubbish.

In hindsight, knowing what I know now, this was a worthless task.  It was a combination of a stupid caching policy/bug in firefox and my uploading "placeholder" images before I had the final design for the favicons. The problem would not have been visible to the client because it was in my local browser cache.  But at the time all I could see was an error!

When is a debugging task "worthwhile"?

I have a long history of tenacity with debugging... and in hindsight many of the more intractable bugs now seem to be examples of time wasted and needless distractions.  Others were important and meaningful battles that I needed to fight to get projects over the line. All were bugs.... are there any clues to differentiate between valuable battles and needless distractions?

When the bug is in someone elses code... 

I think the first possible clue is where the bug lies. I think debugging is often a loosing proposition, even if you do pin it down.  I have spent way too long proving, documenting and reporting bugs in products that I use in the hope that they will be fixed.  Some have been... but I think that the majority of them resulted in a discussion with the product programmer/support minion that not only wasted more time, but resulted in them not resolving the bug in any way that resulted in me getting a fix when I needed it. (I.e very soon after identifying it)
I would suggest that while these activities were probably noble they were essentially of no value to me or the work I was trying to get done. (Except where they resulted in me clearly identifying that the bug was not in my code)

My point?  Stop debugging other peoples code where possible.  There is just not enough time in my life to fix all the broken software (and get a usable result) Once I see that the bug is not in my code... give up.  Find a work-around, hack it, abandon the tool/framework/etc... but move on.

When there is more than one way to acheive a result...

The difference between "the right way to code" and the "expeditious way to code" can sometimes be clear.... however, I find there are usually lots of ways to get to the same result.  Unfortunatly, I often find that the "recomended" way in the docs/tutorials/book/forum post/back-of-the-toilet-door is either out-of-date, incomplete, partially thought out or just plain wrong.  The time I have spent trying to get stuff working that I found on the internet... that eventually turned out to be rubbish, is painful to recount.  (This is mostly my own fault for trying to work in too many languages/toolkits/frameworks etc without taking the time to be an expert first)
I think that I need to be a bit more willing to abandon broken code a bit faster than I currently am. This often comes back to some time pressure, that I'm trying to get something working quickly rather than taking a bit more time to learn the docs first.  False ecconomy.

That being said, I would say that more than 50% of the time when I have gone to the docs for whatever I'm working on, the docs are wrong in part or whole.  So, I'm not sure if there is any actual win with reading the docs first... but it might help in some cases.   To paraphrase... go back to the source Neo...

My advice?  Don't hold onto any ideas of the right/clean/ideologically sound/best practice way of doing something.  Get it working as best you can and clean it up later if it proves to be a problem.  Future proofing is a never ending piece of string argument.... you cannot win, so don't fight the battle.

Usually I find that the next time I try to solve the same problem, I inevitably have a different approch anyway... so I tend to end up rewriting rather than fixing... but thats me.  The key point is to let go of something that is not working no matter how ideologically "right" it may be.

The curiosity trap...

Debugging to figure out how something works is a really painful way to learn.  There are probably lots of scenarios where its the only way (security work, reverse engineering malware, deep bugs in critical systems etc)  but for most of the desktop stuff I do, there is just no reason to be trying to learn that far down the levels of abstraction. It's just too slow.

My point?  I think when its curiosity driving the activity and you find yourself trolling through code... "you're doing it wrong".  Go feed your brain on wikipedia or get a pet or play Sudoku.... the urge to solve puzzles can lead programmers into some really pointless time wasting.

Neat and tidy.... 

The urge to "finish" something or make it "tidy" is a seductive beasty!  It can lead you to make beautiful things... or take you to crazy places.

I find that neatness in coding is important so you can stay on top of the project... but there is a slippery slope beyond that point where neatness for the sake of it becomes a procrastination exercise (he types on a blog....laughing ironically as the keystrokes land...)

Basically, there is no limit to how far you can take neatness in coding.... I think the best advice is to go the other way... encourage messiness to the point you cannot function... then take a tiny step back. Minimize how much extra time you need to spend on the housekeeping...

This ties in with our human pattern recognition systems so it can be a bit of a two edged sword ( triple edged swords just mess up this metaphore...so just don't...)  in that it can be very valuable but also lead to the darkside.

Neatness in the code allows us to scan repeating structures in ways that don't involve fully concious "thought".  This can be a really valuable behaviour; that personnally has spotted issues more times than I could guess.  However... the other side of this problem is that it only works with repetition.  To get maximum value, you need repetative patterns.  Once these patterns get larger... you start to run into the bad 'code smell' of  DRY.  (Don't repeat yourself).   So, I think its probably best used for "small stuff". Such as how I order operators or how I use whitespace in a statement.  The ways that brackets are laid out (this may be why code formatting is such a universal issue for code jockeys)... but once I start to see repeating blocks of statements... its time to refactor or shop for a red lightsaber. 

I think there is a bit more to think about in this area of pattern recognition in coding, but thats another post for another day.

In summation your honour... 

Estimating is always easier in hindsight.  Knowing when to let go of a task is an np hard problem.  Having written all these ideas down, its still not clear that there is any generalisable wisdom in any of it.  But, I feel a bit clearer having articulated some of the issues, which is really the point of writing it down....

Why is Microsoft a bad shepherd?

http://winsupersite.com/windows-8/fixing-windows-8-advice-users-and-microsoft

The above article is yet another mild... how to fix windows 8 piece.  Quite nice and makes some useful points about customising win 8 to suit different users. But thats not the idea that I found interesting.

The seed came from the first comment.  "Microsoft should buy startdock. Or steal some of their employees".  This is both blindingly obvious and subtly interesting.

But why would that be a bad thing?

Consider this,  Microsoft purchases Stardock... or any other company that is building interesting products.  Lets not argue about how much or what the finacials are... lets be real and say that if the decision was made to purchase... it could be done.  This is not the point. Lets focus on what happens.

Microsoft and the mangers of the inhaled company integrate the staff and processes with MS and merge them into one of the MS business groups. They let go anyone that is not right and add some resources where needed.  Imagine it all goes well. (Not making any inuendos... just skipping past distractions while I get to my point) So whats the problem? Everyone is happy and productive.  There is only one thing that has been lost in the process.

Independance. 

The new additions to the Microsoft family... are now part of the Microsoft family.  They are goverened by the same internal politics that have generated the Microsoft platform.  They no longer have the choice to "have a different vision".  They cannot "fix things" that Microsoft do not see as broken. They cannot be a dissenting voice. (They can internally... but it carries less weight in the market place than offering an actual product that provides a solution users want)

The erosion of dissent and the aggregation of control are the things I see as being the death of all great organisations.  As more and more central control takes over a platform, there is less flexibility in thinking, less ability to adapt and address different users needs.  There is more movement towards a shared vision... the so-called "reality distortion field". 

What Microsoft and all the other lumbering giants of the tech industry need is an eccosystem of "loose" collaborators.  Companies, developers and users who all work on the same platform... but with different visions and objectives.  They fill the eccosystem and flesh out all the tiny little niche opportunities. 

Bringing the successful ones under the same vision and managment is just foolish.  Imposing control is the last thing that Microsoft should do to the ecosystem. Their role is simply to foster the ecosystem... to increase opportunity for the benine population and limit the opportunity for the predator population.   They are curators for the platform and the herds that browse upon its bountiful slopes.  Trying to domesticate the herds and put them into factory farms is just totally missing the point.

But, luckily MS have not bought Stardock.  They have not crippled the voices of dissent. Either intentionally or accidentally, there is still a healthy ecosystem around the platform(s).  This in no way forgives the many many many missteps that MS has taken and the many ways that it regularly alienates the ecosystem... but change is always painful.  Some will win, while others will sit around and whinge in the dust... such is life.

Finding my motivation to code again

I have been wonding about what to do to re-invigorate my love of coding.  I figure other peole have the same problem from time to time.

I have just come of the back end of a hecktic year of many small projects.  Because they are "work", there is only so much personal investment that I have in them.  I do them, and I do them well... but it's mostly repetative and ... well work.  It's not fun. Very little is novel.  There is little creativity or problem solving.

I find that after a while, this really sucks me dry as far as motivation to pick up a project when the pressure is off.

So my question to myself was, how do I find my passion again?


The first solution I tried was playing some games.  Honestly, most of the games seem very much Me-too type things that remind me of games from a decade ago.  The thought of diving in a fixing broken game play or hacking on yet another creation kit for yet another half assed clone game... blah!  These are solved problems... they are not even very fresh problems.

Brain training games are even worse... (since I kinda work on them all year anyway... ) they are utilitarian but not ... inspiring.  

The next thing I tried was some little coding competitions.  These were nice.  They got me back to my roots of solving small bounded problems quickly.  It was a challenge to do it and it forced me to use some basic coding skills that have been getting rusty through disuse.

The best problem set I have looked at so far are from Google Code Jam.  They are quick and bite sized.  The data sets are well developed and the online judge does not give you any hints.

http://code.google.com/codejam/

I have found a few more online but have not yet explored them...

https://facebook.interviewstreet.com/recruit/challenges

http://community.topcoder.com/tc

http://www.spoj.pl/tutorials/

http://uva.onlinejudge.org/index.php?option=com_onlinejudge&Itemid=8&category=3


The last thing I have tied is doing something old in a new way... I have started working my way through all the Kahn Academy videos... refreshing, renewing, re-finding ideas and concepts that I have not had to talk about or explain for ... well, a long time.  It's nice.  Kinda familiar, kinda half remembered.  A good refresher...

Maybe there is something there for you?

Brett Victor - Inventing on Principle Video

http://vimeo.com/36579366

This is an interesting perspective to present.  It's also a great way to showcase some very inventive concepts.

Much to think about.

Article on Genius thought processes

http://www.creativitypost.com/create/how_geniuses_think

This is interesting and in some ways ... kinda stating the obvious.  Perhaps a more interesting question is not how Geniuses think... but how everyone else thinks; and why. 

Is thought patterns and strategies a spectrum?  Is there some tipping point between unusual thought patterns and everyone else?

Whats at the other end of the spectrum?  Anxiety? Compulsive behaviours?  Stress? Nightmares?
I think its limits.  That is, either self limiting thought patterns ( I can't draw) or looping thought patterns that endlessly cycle (post traumatic stress) recurrent nightmares etc.  I would also toss in issues like peer pressure, social conformity, body image and other "norming" thought processes that limit options and exploration of possibilities.

The spectrum between these two positions is probably "Normal thought"  which is a little bit from column A and a little from Column B.  


I always find this kind of process interesting because it ends up with me asking myself what sort of limits I impose on myself?  Are these limits real or imagined.  Are they still relevant.

Some are biological,  physical limits due to health and fitness, others are cognitive, some may even be maladaptive.... others are social and finally some are due to phsical resources (time and money mostly) and environment.   There may be a couple of other categories ( legal, saftey, future planning, heritage, family, religious.... that I am ignoring right now.)

The question is, can we remove some of these limits?  Are some of them "Improvable"?  Do we need to swap one for another?  Is it simply a case of doing something differently or is there a trade-off? 

Isn't life fun...


 

Software professional code of ethics... chortle

http://elegantcode.com/2012/04/18/software-professional-code-of-ethics/

This is an interesting article... in spite of itself.  The author makes some interesting points about the futility of the exercise.  Simply because, I feel, they have wandered into the fantasy land of "higher purpose" type ideals. 

I think that any statement of ethics should firstly come from a realistic place. 

1) Look after yourself first.  ( I think self interest must be the basis for any healthy belief structure... but thats me)
2) Manage your clients expectations. 
3) Communicate only as much as your clients will find useful.
4) Design the system so its cheap to change.
5) Design a staged development and implementation cycle so neither yourself or the client has any unpleasant suprises.
6) Establish a fixed price quote for each stage before the work starts and put in place a payment schedule.  (Rolling costs are just a scam.)
7) Manage your motivation. (Celebrate regularly)
8) Write good documentation. 
9) Never trust your client.
10) Accept that everything changes.  Your work is ephemerial.  Nothing you do matters in the long run. Try not to get too attached.

Org psychology at Universities

Within a corporate organisation, you have individuals and the organisation as an entity.

The individuals are working both, for themsevles and to further the goals of the entity(we assume).

This is a nice simple picture... but what if there is resource competition within the organisation?  Say inter-department competition for resources? Is it just an inverse hierarchy? Simply an us and them fight only differentiated depending on the level at which the competition occurs?

Ok, fairly simple so far....


Now what happens if the managment is cash-cowing one department to expand or push resources into another? 

At the indivudual level.... probably not a huge change. (0)
At the department level... one is asked to altruistically "take one for the team".  (-1)
At the organisation level.... progress. (+1)

Now what if the organisation is a University where staff not only have loyalty to themselves, their department and their University... but also to their field and professional organisations. 

Suddenly taking one for the team means that their department, field and professional organisations are robbed while resources are sent elsewhere.  Research they are passionate about and think is significant is stalled or blocked while someone else eats the cake.  Keep in mind that a researcher who is not researching... is effectivly wasting their life.

So, seeing resources moved elsewhere for "organisational purposes" results in something like this

At the individual level .... probably no change (0)
At the department level .... a loss (-1)
At the organisation level ..... a win (+1)
At the field level .... a loss (-1)
At the research level ... a loss (-1)
...etc

Universities are more like charaties that support people... as soon as one stops paying... the people need to move to another.  There is no point sitting and marking time.  Time is the most precious resource.

Analystics for software groups

http://radar.oreilly.com/2011/10/moneyball-for-software-enginee.html

This is an interesting application of the same principles in moneyball to software teams.

No more sysadmins?

http://peebs.org/ill-probably-never-hire-another-pure-sysadmin

This is a noteworthy post simply because it illustrates change.  Adapt or die kind of thing.

Stupid as an asset

http://jcs.biologists.org/content/121/11/1771.full

 This is an interesting article that positions stupid as an essential aid to good science... or part of it anyway. Also suggest that lawyers don't get challenged... but that's just an aside.

Teaching Programming

http://evincarofautumn.blogspot.com/2011/09/tricky-programming-concepts-arent.html

Teaching programming concepts and philosophy.

Career Advice for Coders

http://www.kalzumeus.com/2011/10/28/dont-call-yourself-a-programmer/


This article is well worth the read, just for the writing style. The content is excellent too.  The interesting point is just how it reflects on a coder working in an academic environment.... which essentially has no real profit motive and only vague concepts of "reducing costs".  This means that making any value proposition about yourself has to be couched in the currency of the organisation... be it research or teaching.

The other upshot is that, as the article suggests, academics have no money and offer really crap pay.  They value other "currency", such as publications, grants, research, awards etc.  Probably because the monetary returns of academia are fairly stable, not particular high and fairly well known. It's really only the "super stars" of the academic ( or uni administration) who make a reasonable living.  But, funnily enough, they still have to argue a value proposition, much like the above article outlines.

Essentially, apply the ideas in the article or take what the world hands you.  Control your destiny or be controlled.


On a similar theme is this post

http://www.sarajchipps.com/2011/10/developers-and-professionalism.html

Which also points out the obvious (once you grok it) that the clients perception of you relates directly to the amount they value you. As I said ... obvious.

http://www.jacquesmattheij.com/I+am+a+programmer

On the same topic, and illustrating some of the same points but from a slightly different POV.

Ugly Code?

http://www.gamedev.net/blog/355/entry-2250790-there-are-many-kinds-of-ugly/

(This is the bakery reference in the above linked article http://www.joelonsoftware.com/articles/Wrong.html )


This article makes a valid point about "the eye of the beholder" issue with code uglyness.  I would make two extension points.

The first is that ugly code may also be a result of code reflecting an ugly problem domain (code in a swamp).  The second would be that it may reflect the limitations of the tools or the staff using the tools (Blunt instruments). 


Coding in a Swamp

Very few problem domains are neat.  Its fairly easy to build an elegant solution in an elegant environment ( such as within the Unix context) however, once you get out into user land with the safety off and profit motives on... mostly people only have half a clue about what they are trying to do, how they want it done, what the consequences of doing it might be and how much they think it will be worth if they have to do it again.  Now multiply the number of users by whatever factor you like and the complexity of the problem domain scales exponentially in so many dimensions it makes your eyes bulge just trying to bound the problem space.  I pity the poor fool who imagines that they can find one neat architecture that will elegantly describe this order of complexity. 

My point being, ugly code in this context is really a reflection of the programmer(s) evolving understanding of the context... its unpredictable, uncertain and often moves in weird cycles and feedback loops involving users learning the tools, which allow them to gain a better understanding of their own environment, which leads them to criticize the tools as being inadequate to describe their requirements... this feeds back to the programmer in "user speak" who then needs to re-imaging a solution in light of their often contradictory, partial or just plain weird information.  The worst part being that the "user speak" is inevitably phrased in terms of "variations" on the semantics of the tool that exists in front of them at the time they made that comment... along with their memory of any variations of the tool that have gone before.  This leads the programmer to see the users request as a variation on what already exists rather than a new framing of the original problem.  Thus we get patches and tweaks on the semantics of what already exists which wanders further and further from the original anchor point of a clean design.

This process never ends.  There is no point at which you get to stop and re-start the project... production code does not die... but some things "harden". Some ideas crystalise and everyone stops finding tweaks and fiddles that they want.  This is the opportunity to factor that "Idea/semantic/feature" out of the mud and clean it up.  While around it the rest of the primordial soup continues to evolve in ugly fits and starts with lots of culling of the weak designs.



Blunt Instruments

The other cause of ugly code is code produced by an ugly process.  This can be poor tools, poor processes, poor staff, poor managment, lack of resources etc etc etc. In summary, code that does not have a low cost to change.  This means code that is expensive for whatever reason, to make better. 

Unfortunately, we exist in an environment that is for all intents and purposes... ugly.  It has rough edges, obstacles and failures. We exist in an imperfect world and thus our code reflects this context. Either accept or face frustration and insanity.  You can argue as much as you wish, make excuses, spend the future or take to drink... whatever helps you cope. 

Evolution is at work on you... and chances are you are not going to win....

You're a bad programmer...

http://badprogrammer.infogami.com/

There is much wisdom and a little sarcasm in this article. Must read for people making the transition from amateur to journeyman programmers.

"Whats wrong with this sentence" or my rant on novelty in my work...

I found a sentence in a blog post that bugs me. The post is interesting and on another topic, its just this one sentence that irritated something that has been cooking for a while... (Just to be clear, this post has nothing to do with the other post and does not comment upon the other posts, content, author, style etc.  It's just been triggered by a juxtaposition of words that happened to occur in the other post.  Ok, the sentence follows...

"We're also not looking to add to our team, and if we decided we did want to grow I expect we wouldn't have any trouble finding someone who was looking for a well-paying job in manhattan using very cool technology. "( From "Life after Pair Programming" by Jay Fields)


Why does this sentence bug me?

1. A programming job is about location, pay and tools/technology?

AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH.  I've seen this sentiment in a number of places... but at its heart it points out the rotten issues at the heart of so many IT (other) jobs.  No passion.  How can you work on a job that you are not interested in the content first and foremost?  Or at least willing to talk about it first and foremost?  While its nice to have location, pay and technology... these are side benefits. (true they may tip the balance and rightly should... ) but not really mentioning the topic, subject or content of the task seems to be suggesting that its really not much of a highlight.

2.  Perhaps this sentence illustrates whats wrong with my job?
Is the content in my job really that .... note worthy? Is this, like everything else, fundamentally about me? (Probably...)  Am I just unhappy, grumpy, pissed off, tired, sick and seeing the meaningless repetition in my work?  Hell yes...

Does this mean that I have mastered everything in my world? Mostly.  I think the kind of surprises that show up in your work are very illustrative about your level of mastery. 

How novel are the problems you encounter? I have constructed a scale... from not novel to really truly novel. The scale is based on three dimensions.  (Experience with solving, Tools to solve, Domain Knowledge about)

The Experience dimension

This moves from ... "Its all new" to "I remember solving this yesterday".  Essentially, its about how much your memory is involved in the solution.


The Tools dimension

This dimension moves from .... "No currently known process to solve this" to "I have a debugged script that will do this with a single button click".  A simple metric in IT land would be the number of mouse clicks/keystrokes required to affect a solve.

The Domain Knowledge dimension

This dimension moves from .... "I  have only the general working vocabulary to start to talk about/search for this topic" to "I can describe it using 3 or less key words and get a solve in the top couple of pages on " .  Once you can explain something, deeply, succinctly and thoroughly to someone else (or to a search engine)... you own it. It's no longer novel or hard.


So how does all this come together?

Well you build a table in excel, record a set of problems that arrive in a give time frame and score those suckers on the three dimensions above.  Use a 5 or 7 point likert scale for fun and see how novel your job really is.  You can repeat this sampling strategy at various intervals and test hypothesis such as

H0: My job is getting more novel.
H1: Nope.. I'm getting more experienced.

(You should see the experience scores going up while the others stay constant)

H0: Coffee is making me stupid.
H1: Nope.  I was this stupid all along.

(You should see all measures stay constant. Combine with varying (and recording) your coffee intake over time.)

Many other fun experiments can be done using this scale....

But really, whats my take away from this all.... mainly that I'm feeling sick (true) and miserable and probably should not design experiments to evaluate anything in this condition..... shutting up now.

Best Practices and Philosophy for the Developer

http://www.artima.com/weblogs/viewpost.jsp?thread=331531

There are not too many new nuggets here but its a good roundup of lots of the most quoted gems for software engineers.

Survival Guide

http://www.xamuel.com/mathematicians-survival-guide/

This is worth a read for anyone in academia; especially the more rigorous science fields. 

Software engineering proverbs

http://www.multicians.org/thvv/proverbs.html

There are some good thoughts on this page.

Yet another programming gem

http://asserttrue.blogspot.com/2009/03/how-to-write-fast-code.html

Go faster by doing less.  Obvious once you conceptualise it and articulate it.

Master Programmer Post

http://www.zedshaw.com/essays/master_and_expert.html#

This post explains the idea about moving toward simplicity as a property of mastery.  I agree.