Friday, June 23, 2017

Progress Bar For dd(1)

The dd(1) command is frequently used to both make and write out image files, such as .iso files for Linux releases.  It's mostly intended for this and works fine, but a common complaint from users is the lack of a progress indicator.  It's true:  dd is silent until it's complete.  But you can get a progress bar with the pv(1) command.

pv is short for Pipe Viewer.  You can get it over here or install it from your distribution's packaging system.  For example, Fedora users can do dnf install pv.

Once you have pv installed, you can start getting progress output for console commands.  The man page has some good examples, including one for tar(1).  The man page also suggests using pv by itself as a replacement for dd entirely, but I have had limited success with that.  Here's what I do:
dd if=ISOFILE | pv -p -e -a -s BYTES | dd of=DEVICE bs=4096
OK, that's a lot, but it's not that bad.  dd reads from stdin when not told otherwise.  Likewise, it writes to stdout when not told otherwise.  In this example, we connect two invocations of dd using pv, which also has the same behavior.  By not specifying either on pv, it reads from stdin and writes to stdout.

The ISOFILE is the name of the image file I want to write, such as release-3.2.0-ppc64-20170623.iso.  BYTES is the size of the image file in bytes.  This is obtained any number of ways, such as the long listing from ls(1).  DEVICE is the device node you want to write the image to, such as /dev/sdh.

You could extend this and have the command determine the size of the ISOFILE directly, such as:
dd if=ISOFILE | pv -p -e -a -s $(stat -c %s ISOFILE) | dd of=DEVICE bs=4096
Now this is starting to look like something you could put in to a helper script in your own environment.  Just another tool for your command line adventures.

Sunday, June 18, 2017

Did Someone Say Trains?

No?  Well, I guess I did.  My new commute has me taking three different lines on our transit system to get to work.  It reminds me back to the early 2000s where I would constantly give friend after friend updates on the fact that MARTA is single tracking for the day.  Like anyone cares.

Last week's heat wave caused some hiccups on the MBTA.  The Internet fortunately gives us a venue for all of the experts to offer up comments on what is wrong with the system when anything out of the ordinary occurs.  So what happened that I noticed?
  • Green Line power surge near Kenmore Square.  Since it was a heat wave, my unscientific guess is that people were firing up air conditioners, which impacted the grid.  I only say this because it's happened before both here and in New York.  My intern was affected by this on his first day in the office, but it wasn't really a major thing.
  • Medical emergency on the Red Line.  Well, these tend to happen daily, but I think on the heat wave days there were more.
  • Trash fire on the Red Line.  This is the best one I heard about but still not better than the time I heard on the Green Line radio an operator calling dispatch to report a huge turtle on the tracks (different day, different year).
  • Slow commuter rail.  Likely a heat order from dispatch, but I am guessing.  In extremely hot weather, continuous welded rail can buckle or kink which can lead to a derailment.  This has happened on multiple occasions for both passenger and freight train.  A heat order [usually] requires passenger trains to go 10 or 20 mph below the maximum allowed speed.  The slower speed is meant to help engineers spot rail problems and report and avoid them.

I was only involved in one issue last week, not noted above.  I was going to catch the 5:50 pm outbound train from North Station.  We boarded and then at 5:55 pm, a conductor told everyone the train was out of service because the locomotive wouldn't move (breakdown!).  So we all had to detrain and retrain (crosstrain?).  That took about half an hour.  As we left the platform, we were not in time to beat the Amtrak Downeaster so we had to stop for it to pass.  We continued after it passed, but were traveling at medium speed.

People were very angry about the entire thing.  And yeah, I get it.  Karen was waiting for me at Bistro 5 and our plans changed up, but whatever.