I decided to experiment with something I had been avoiding for a while. Buying books off Amazon. I have been avoiding it because having done it in the past a few times with very poor results, I had considered the whole experience to have such a low success to failure ratio that it was not worth the effort. Even with the illusion of low prices.
Anyway, I tried it again, on the rationalization that ... perhaps things had changed.
So after placing a couple of orders for some tasty looking reading... I waited. One of the books arrived after just a week. This is equivalent to Amazon implementing teleportation. Suffice to say I was both impressed and anxious to open the package.
Inside was a book and a packing slip with the name of the book I had ordered in very big letters. Naturally the actual book delivered did not match the packing slip. It was instead a 19 year old office administration text book with no residual value that I can detect except to a historian. (Maybe?)
So I complained and asked for the correct book from the seller. Instead I got a prompt refund and a message from the seller saying that they had checked their stock and did not in fact have that book at all and I should keep what they had sent and not bother to send it back. Ya Think!
So I tried to follow the train of though of the bookseller. They get an order through Amazon, and check their stock database? They find an entry for the book ( obviously otherwise they would not have even entered it on to Amazon to sell... I assume) So they print a packing slip and hand it to the picker/packer who was doing the packing. The picker gets a packing slip, wander to the stock shelves and look for the book, which they don't find(or do they)....
This is the point where the thought train goes in one of two directions...
The fist, the picker who has not found the correct book, instead decides to send what is obviously rubbish, packs it and dispatches it. Why? Were they hoping that I would not notice and instead accept the substitute without complaining? (Playing the odds? Seems pretty unlikely) Or instead were they trying to delay something? Perhaps so they can tell Amazon that the item has been dispatched and so keep up some sort of statistic? Either way, this results in their company wasting money(shipping and price of one book) and wasting my time.
The second, is that the picker likes the look of the book they were supposed to send instead sends some crap that would never sell anyway while they steal the actual book for themselves. Again wasting their companies money ( shipping cost and price of the two books) and wasting my time.
Either way the book picker has wasted their companies money and my time. Not factoring in the time and effort of their staff to respond to my email, check the stock rooms, update their database, and feel pissed off that they are going to get bad feedback on Amazon. Essentially their book picker should be fired.
If only they had actually confessed that the book could not be found and had send a kill order back through the system it would have saved time and money for all concerned.
Now I just have to go and bitch about them on Amazon. Ah the little pleasures.
Saturday, October 30, 2010
How to fix broken or incomplete torrent downloads
Just taken the time to figure out how to fix torrents that go dead or are incomplete due to running out of seeds or peers with enough pieces to complete them.
The problem
One of my torrents was down to one peer and we both had the same pieces.
My option was to kill the download and try a different torrent or find some more peers with the same torrent hash. The only way to do that was to find some other trackers for this same file hash. Also, I didn't want to leave the old peer hanging, so I needed a solution that would merge a number of torrent files together somehow.
The Solution
1. Hit google and find every other torrent file for that particular file/file set you were downloading. Make sure the hash is the same.
2. Crack each one open in http://torrenteditor.com and check if there are any other trackers that are tracking the same file hash.(Note the piece size as well just for interest sake)
3a. Either add the tracker URL's to your existing download ( you can do this in Vuze. I have no idea about other BT clients) using copy and paste (In Vuze you need to right click on the torrent > Advanced > Trackers/Torrent > Add Tracker URL.
3b. Or use TorrentEditor to create a new mega torrent file containing every tracker you can find, then import it into your client and point it at the directory that you were downloading into. Vuze will re-check the file and pick up where it left off. (Assuming the piece size is the same. If its smaller you may lose a little but if it works finish an otherwise dead dl)
5. Cross your fingers that you have found enough seeds/peers to complete the file.
6. Remember to keep seeding after you have finished to help others.
Nothing like frustration to motivate a solution to an ugly problem.
This should be filed under ideas like bridging trackers and merging torrent files. Hope this helps someone else.
The problem
One of my torrents was down to one peer and we both had the same pieces.
My option was to kill the download and try a different torrent or find some more peers with the same torrent hash. The only way to do that was to find some other trackers for this same file hash. Also, I didn't want to leave the old peer hanging, so I needed a solution that would merge a number of torrent files together somehow.
The Solution
1. Hit google and find every other torrent file for that particular file/file set you were downloading. Make sure the hash is the same.
2. Crack each one open in http://torrenteditor.com and check if there are any other trackers that are tracking the same file hash.(Note the piece size as well just for interest sake)
3a. Either add the tracker URL's to your existing download ( you can do this in Vuze. I have no idea about other BT clients) using copy and paste (In Vuze you need to right click on the torrent > Advanced > Trackers/Torrent > Add Tracker URL.
3b. Or use TorrentEditor to create a new mega torrent file containing every tracker you can find, then import it into your client and point it at the directory that you were downloading into. Vuze will re-check the file and pick up where it left off. (Assuming the piece size is the same. If its smaller you may lose a little but if it works finish an otherwise dead dl)
5. Cross your fingers that you have found enough seeds/peers to complete the file.
6. Remember to keep seeding after you have finished to help others.
Nothing like frustration to motivate a solution to an ugly problem.
This should be filed under ideas like bridging trackers and merging torrent files. Hope this helps someone else.
Friday, October 29, 2010
The effect of traffic roundabouts on driver attitudes
Had an idea on the way to work this morning. Traffic roundabouts are training drivers to be more opportunistic and proactive in contrast with traffic lights which train drivers to be passive and rules bound. (Also frustrated due to the lack of control and self determination)
I wonder what effect this has on their satisfaction with life beyond the road?
The perception of control is also interesting, in that generally (on a flat roundabout) you can see everyone and know where you are up to in getting to where you are going. While with traffic lights, you never quite know if you are getting a fair deal.
Moving on...
I wonder what effect this has on their satisfaction with life beyond the road?
The perception of control is also interesting, in that generally (on a flat roundabout) you can see everyone and know where you are up to in getting to where you are going. While with traffic lights, you never quite know if you are getting a fair deal.
Moving on...
Profiler for VBA code in Access
I found a nice little stack style Profiler for VBA from a book called Access Cookbook. Its been very handy for identifying and tracking down a couple of memory leaks (My fault as usual) The Profiler is in a module called basProfiler and is discussed in an article called "Create an Execution Time Profiler" which has been reprinted all over the web. I had to fix a small bug in the code that formatted the output to the log file. Other than that it was good to go out of the box.
My usage of basProfiler
1. Import the module.
2. Set the path to the log file to somewhere conveinent. (I put mine into my working directory for subversion to manage)
2. Instrument the code blocks with the opening and closing lines.
I open the log file in Notepad++ on the other monitor and every time I run the code, Notepad++ detects that the file has changed on disk and allows me to reload it. This gives me the power of a real text editor rather than a log window to play with the output.
Simple and elegant. Just the way I like my tools.
I considered hacking it to output a comma delimited file for import into excel for analysis but I ended up not needing that sort of power on this job.
And we are done.
My usage of basProfiler
1. Import the module.
2. Set the path to the log file to somewhere conveinent. (I put mine into my working directory for subversion to manage)
2. Instrument the code blocks with the opening and closing lines.
Public Sub Whatever()
acbProPushStack "Whatever" 'profiler
'code to be profiled here
'.....
acbProPopStack 'profiler
End Sub I open the log file in Notepad++ on the other monitor and every time I run the code, Notepad++ detects that the file has changed on disk and allows me to reload it. This gives me the power of a real text editor rather than a log window to play with the output.
Simple and elegant. Just the way I like my tools.
I considered hacking it to output a comma delimited file for import into excel for analysis but I ended up not needing that sort of power on this job.
And we are done.
Quicksort for VBA array of custom objects
Can you believe that there is no sort function for the collection object in Access VBA? Anyway, I have dumped the collection for a typed array to avoid some of the overhead of type casting variants everywhere. (Long story involving a profiler and a lot of time which finally resulted in finding a different source for the bug that was causing the slow down.)
Back to the main thread. I went looking for a good implementation of a sorting algorithm in VBA. After a couple of absolutly spectacular crappy implementations I found this one.
http://www.java2s.com/Code/VBA-Excel-Access-Word/Data-Type/QuickSort2.htm
Which is short, simple and elegant. In comparison to some of the other multi-page examples I found its really ... beautiful.
Anyway, the first thing I did was refactor out the comparison operation and the swap operation, so I could apply the operation to an array of custom objects without adding clutter to the quicksort sub. By extracting the comparison operation, I can now sort the objects in the array on a variety of properties. If I get bored I will implement both ascending and decending sort but I don't need it right now.
The most painful part of this is not being able to write a template for the code and make type independent. (Anyone mentions VBA variants as being equivalent and they are outa here ...)
Just thought I would share...
Back to the main thread. I went looking for a good implementation of a sorting algorithm in VBA. After a couple of absolutly spectacular crappy implementations I found this one.
http://www.java2s.com/Code/VBA-Excel-Access-Word/Data-Type/QuickSort2.htm
Which is short, simple and elegant. In comparison to some of the other multi-page examples I found its really ... beautiful.
Anyway, the first thing I did was refactor out the comparison operation and the swap operation, so I could apply the operation to an array of custom objects without adding clutter to the quicksort sub. By extracting the comparison operation, I can now sort the objects in the array on a variety of properties. If I get bored I will implement both ascending and decending sort but I don't need it right now.
The most painful part of this is not being able to write a template for the code and make type independent. (Anyone mentions VBA variants as being equivalent and they are outa here ...)
Just thought I would share...
Friday, October 8, 2010
Thoughts on this years crop of research student projects.
Thoughts on this years crop of research student projects.
The projects are not unusual. The same sorts of mistakes by the students trying to get up to speed quickly with complex systems and subtle processes that take time and experience to master. Issues like
These are all throw-away systems so are not being maintained in any way. This year has been remarkable for the size of the data sets I am being asked to work with. The use of automated data collection systems is allowing researchers and student to collect greater volumes of data. Some of which I would suggest is never going to be used and just adds bulk to the task of transforming the data sets and stresses the tools.
This is again a mistake from ignorance. If the students had done the experiments on paper and then hand processes all their data I think they would be a little more constrained when they next planned an experiment. Is this a good thing though?
While the downside of automation is that an inexperienced researcher can generate huge amounts of worthless data, it also allows an experienced researcher to "Think Bigger" and not feel constrained by the hard lessons they learned as an undergraduate that have limited their scope in the past.
I still have issues with people trying to keep a foot in both camps. Who are using the automated tools to generate massive data sets and then trying to hand processes them. This ends up being the worst of both worlds. There is a project going on at the moment that has gone down this road and is reaching epic levels of manual labor. Essentially all the data is coming from databases, being converted to static reports and then hand entered back into a new database. All without any intentional transformation of the data. And its a huge data set containing more than 1600 items per record. And did I mention they are trying to code it on the fly using an evolving code book so it can kind of end up in an SPSS file like they used to do in the old days... Talk about frustrating. Its going to take months of work, thousands of man hours of labour and introduce all manner of human errors into the data... all because the lead investigator has control issues and cannot grasp the scope of the problems they create for everyone else.
Ahhhhhhhh.
But back to the students. Like most years its been interesting. There have been a constant flow of new problems to solve and an acceptable number of repetitions of previous years problems. Some of them are starting to get a little tedious but I am still learning new tricks so its not a total waste of time.
I am encountering a couple of recurring issues that just add to the misery. One is dependent questions in Surveys. The other is designing projects to manage attrition of participants. I have a paper in mind on the first issue but I have not really decided how I want to try to deal with the second. Its still evolving in my head. Maybe a blog post will let me work out some angles....
Later.
The projects are not unusual. The same sorts of mistakes by the students trying to get up to speed quickly with complex systems and subtle processes that take time and experience to master. Issues like
- Not testing their experiments completely before starting to collect data.
- Not keeping a log of their experiment activity
- Not having any idea about how to process data after its been collected
- Having no quality control concepts
- Having only the most basic ideas about backup and versioning
- Designing experiments with no resilience or risk management concepts
- No time management skills
- Not being computer literate and running computer based experiments(..words fail me!)
These are all throw-away systems so are not being maintained in any way. This year has been remarkable for the size of the data sets I am being asked to work with. The use of automated data collection systems is allowing researchers and student to collect greater volumes of data. Some of which I would suggest is never going to be used and just adds bulk to the task of transforming the data sets and stresses the tools.
This is again a mistake from ignorance. If the students had done the experiments on paper and then hand processes all their data I think they would be a little more constrained when they next planned an experiment. Is this a good thing though?
While the downside of automation is that an inexperienced researcher can generate huge amounts of worthless data, it also allows an experienced researcher to "Think Bigger" and not feel constrained by the hard lessons they learned as an undergraduate that have limited their scope in the past.
I still have issues with people trying to keep a foot in both camps. Who are using the automated tools to generate massive data sets and then trying to hand processes them. This ends up being the worst of both worlds. There is a project going on at the moment that has gone down this road and is reaching epic levels of manual labor. Essentially all the data is coming from databases, being converted to static reports and then hand entered back into a new database. All without any intentional transformation of the data. And its a huge data set containing more than 1600 items per record. And did I mention they are trying to code it on the fly using an evolving code book so it can kind of end up in an SPSS file like they used to do in the old days... Talk about frustrating. Its going to take months of work, thousands of man hours of labour and introduce all manner of human errors into the data... all because the lead investigator has control issues and cannot grasp the scope of the problems they create for everyone else.
Ahhhhhhhh.
But back to the students. Like most years its been interesting. There have been a constant flow of new problems to solve and an acceptable number of repetitions of previous years problems. Some of them are starting to get a little tedious but I am still learning new tricks so its not a total waste of time.
I am encountering a couple of recurring issues that just add to the misery. One is dependent questions in Surveys. The other is designing projects to manage attrition of participants. I have a paper in mind on the first issue but I have not really decided how I want to try to deal with the second. Its still evolving in my head. Maybe a blog post will let me work out some angles....
Later.
Subscribe to:
Posts (Atom)