More Visual Studio shenanigans

I just received an email with some light detail about Visual Studio 2012 update 3 RC.  After having a look at the lean list of fixes included and the news that MS is prepping for VS2013... well I was pretty disapointed.

Suffice to say that effectivly I have not been able to use VS2012 for much of anything.  Its either been feature incomplete or buggy as shit to the point of unusability.  So apart from not progressing a couple of projects... I will essentially have to skip a whole version before (hopefully..) I can make some progress.  Talk about stealing momentum.

While I understand that VS is an "everything and the kitchen sink" type toolbox and getting it to all work is a herculean task... it seems like this version has just sucked constantly.  Its been such a political release with MS trying to strong arm their agenda for Win8 down my throat that I am just disgusted by the whole experience. 

The sad thing is that Win8 would have been picked up and used quite happily as part of the natural cycle and no one would have mentioned it.  We would have migrated everything in our own time and at a comfortable pace. But the whole "forced adoption" policy just broke too many things all at once and really did not leave me with a path forward.  And that left me holding the bag and trying to explain to a lot of clients why I could not deliver what seemed to be a simple update for a new Microsoft Platform.   You just cannot do that and not expect to cop some flack for it.

Diversification is the only way to manage risk across a broad portfolio.  So now, I have to diversify and spread my position away from Microsoft.  I have to maintain a number of possible solutions on a range of platforms and middle ware rather than relying on MS to be both platform and middleware provider.  The trust is gone.  I have spent more time reviewing tools sets and API's from other platforms over the past year than any time previously.  I have had to develop on a dozen different middle ware packages and push more projects into the cloud that otherwise would have been on the MS desktop.  All because they moved my cheese and pissed all over it.

And for what?  Has the MS position in the market place been strengthened?  Hardly.  I still can't find a supplier in the city who cares about selling surface tablets. There are now just as many Mac products amoung my clients as there are PC.  Chrome books and Android tabs are breeding and market place apps are spawning like crazy salmon.  The monopoly is gone; panicing is just making it worse.

Visual Studio Update fixed flickering bug

Today I happened to start Visual Studio 2012 for an indirect reason and it told me there was an update for NuGet availible.  I let the update install... and then restarted and....   the horrible flickering and endless instability were gone.  

This behaviour has been paralysing my use of Visual Stuido for months.  I guess I can finally get back to the job I have had on hold... once I finish all the others that have filled the vacumme. 

Whoot!

(So, in hind sight... I guess its been the best part of 18 months since I have been able to make meaningful progress on that job due to all the f@#% ups with Visual Studio, Win7/8 and the features that were missing and finally showed up in service pack 1.  While my disgust knows no end... I have been using a bunch of other developement tools recently and know just how polished VS really is... despite some of its crippling flaws)  What a frustrating life we get to lead....

MATLAB Refactoring

Today I hate Matlab. 

My problem is that I have inherited a pile of scripts that have been hacked on by a range of researchers in multiple countries over multiple years to do multiple experiments.... (I hope you get just how ugly that is).  Each of these researchers is a gifted amateur. That is to say, they are very good at hacking in a fairly oldschool procedural style of coding. Which is precicely where everyone starts... no problem there.  The problem is the scaling issue, lack of discipline, quite varied use of idiomatic styles and all the usual code-rot problems of a codebase that has been recycled multiple times without any particular clean-out or clean-up.  Its got gunge!

Where do I start?

Firstly, get my tools lined up. get it under source control, create a working copy...

Tools
Matlab and Notepad++

Now start reading to see what I have to deal with....

As per Matlab convention... they are basically one massive function per file.... some very rudamentary decomposition but still about 5kloc over 4 files.  Shit....

Scads of cryptic variables.  Lots of them seems to be allocated within complex calculations and then used here and there.... then turn into arrays and cell arrays or just get dumped to files where they die.

Lots of magic numbers.....

Lots of one line calculations doing multiple things with multiple cryptic variables and a scatter of magic numbers.

Did I mention the abysmal lack of whitespace.
Did I mention the oldschool style of keeping all names in lower case and minimising the number of characters (looks like they were trying to keep it under 5 characters for some insane reason??/)
Did I mention the lack of meaningful documentation?
Did I mention the lack of any verifiable test for anything?

The only documenation of intent ("Intentional" descriptions) is a bunch of research papers that have been written off the back of this snarl that document what the researchers think they were doing.... which scares me when its very hard to tell if the code actually did that...reliably.

Now to start the refactoring.

Step 0.  Format the Code for readability.

In Matlab (Ctrl+A, Ctrl+I) for the auto-indent function. This works nicely and I have not found any problems with it.

Make sure your functions have the matlab "optional" "End" keyword.

Now where are the styleing tools????  WTF? There are no whitespace tools for matlab... you must be F@#@$% kidding.

Here is the .m file I wrote to fix that.  http://duncanstools.blogspot.com.au/2013/06/matlab-fix-style-tool-to-fix-formating.html

Step 1.  Name stuff. 

Name the functions as meaningfully as you can. 

 This will change as you work through the codebase. But name them to describe their "function" as much as you understand it.

Start naming the variables to describe what they are doing.  

Matlab has some fairly handy tools to rename variables... they are a little flakey and sometimes fail which means manually undoing what you have done.  When this happens I use the find tool to work backward up the file until I find the first instance of that variable and try to rename it at that point. Usually this works.  If all else fails use find and replace....

Step 2. Group Stuff. 

Use the matlab code cells (use a double percent characters with a following space to start a comment "%% " and it will create a different "region" in the code. This is an easy way to "soft" group a region of code that you think is a candidate for extracting into a function.

Start looking for any variables that are "recycled" and split them up.

Start looking at where variables are allocated, initialised and used. Group them together and see if you can package them up into a nice little unit of functionality.

Break out all the repetitious code and sketch out a reusable block that might work to replace them.  Make an class if it will need to hold state data or a function if its just needs some scope.  Beware of Matlabs scope rules... they are a bit weird if you have come from other languages.

Matlab allows you to have multiple functions in the same file.  Just start the file with the "big" function and the put "helper" functions below.  There are no refactoring tools to help you with this but the syntax highlighting and variable highlighting are pretty good and help alot when extracting sections of code manually.

Package up related material and then clean up the packages.  The exercise of packaging is really an exercise of breaking the total "scope" of the code into small managable bits that you can hold in your head all at once.

By driving a structural "fence" through a large peice of code (using functional decomposition), and managing what crosses the fence (passes into and out of functions), you are immediatly simplifying your mental schema. Once you have the system decomposed into chunks of a size that you can hold in your head comfortably, you are done.  Now just look at each chunk and clean it up.   

Step 3. Write Stuff. 

Write comments to document what you think a variable is for.
Write comments to document what a function or group is doing.

The more you write the more clearly the code will come out and the less you need to hold in your mental schema at any point in your coding. This is the objective of all refactoring.  Unload your mental schema of the code so you can get a handle on the whole game. 

Setp 4.  Delete Stuff. 

This is the most pleasant step.  Delete all the crap comments,  bad formatting, irrelevant and duplicate code, old style idioms, bad names, unwanted functionality and anything else you can find that in not being used NOW.  Kill it all.  Make your code shiny and clean.  If you need something later, add it back in... don't try to carry it forward based on a "maybe useful later" kind of logic.  This does not pay off.  Go back to the code in the repository and have a look if you want to refer to something that was in an old version.  Be ruthless.  Get rid of everything that is not "Shiny" and "clean smelling" (to borrow from Fowler et al.)

Does Matlab help or hinder refactoring?

Don't get me wrong, Matlab is a powerful toy with a shiny shell and a big price tag, but its not a productive toolset for larger projects.  Then again, I doubt its intended to be.  So maybe this is just me thinking about it in a way thats inappropriate.

There are some good features built in that provide some basic tools for the refactoring process, but there is little beyond that.

Find and replace Tool.  It's good, but not great.
Block Comment/Uncomment.  It's there and works without mangling the code
Auto-indent.  It's good.
Auto-Style.  NONE.  Check out AStyle or Profactor StyleManager, PolyStyle if you don't know what I mean. Also this article on CodeGuru - MakeCodeNicer by Alvaro Mendez (Took me a lot of effor to find this references. I havn't used this code in 5 years but it sticks in my memory as part of my essential toolkit back then...)

Higher level refactoring.  NONE. Is this a problem or am I just applying Matlab to bigger things that its ready for?

Later all...