Monday, December 24, 2012

Larry Berry (Mintec) 1962 - 2012

This is another somewhat philosophical post.  I promise I will post code and tangible how-to stuff again.

A mentor of mine, the individual who turned me on to Python, has passed on.  The whole deal was a bit sudden and really gave me pause.  What I'll try to do here is briefly summarize Larry's body of work, and put it in perspective with regard to software development and domain specific software.  Although his efforts won't see headlines in either mining circles or software ones, what he did was pretty amazing.

Larry worked for Mintec, a mine software vendor, from 1990 on. His specialty and his passion were Ore Control, the place where mine operations and engineering come together, the place, figuratively speaking, where the rubber meets the road. This part of the mining process, in my opinion, was often the bastard child of most upper management. My impression was that at the time Larry began his career at Mintec not all, but many management types had the idea that if you just planned the mine right and threw a bunch of grunts into the pit with some equipment, everything would magically happen. Tactical execution and measurement of that process are important. Larry was one of the people who recognized this immediately and did something about it. He championed until his death the idea that Ore Control and short range planning were information intensive parts of the mining process that demanded an operational database and a programming API that allowed for dynamic process configuration.

Around the year 2000, Mintec released a planning database and a Python API. Larry was very much a software implementer at the time and not a developer. Nonetheless, his input was an important part of the feedback loop within the relatively small mine software company. He told me that they had been looking at Lua as an API. It did not particularly work for him and they settled on Python instead. By comparison, I've heard that one of the other software mining companies uses Perl for its API, and another one VBA. I'm biased, but I think that for ease of learning for the customer and internal productivity, Mintec made the right choice. They started out with Python 2.1, then, along with some changes to the API, switched to Python 2.2. Today we are using Python 2.5.

As a programmer, Larry was mixed. He was a miner first, and anything that got in the way of his goal was a mere distraction. His code was clean, but seldom bulletproof. I would call him an offensive programmer, the opposite of a defensive one. He was an upbeat optimist. I was and am a somewhat morose pessimist. Much to the entertainment of those working around us, we would argue about code. The Larryism that caused me the most heartburn was the variable declared inside a conditional block (Python is a dynamic language that allows you to do that) that under a corner case didn't get executed.

Larry: Well if your data were squared away it would work!

Me: LARRY! Our data are NEVER squared away!

The crown achievement of Larry's Mintec career, in my estimation, was not so much his code, but his driving through of the whole MineSight Operations package, basically the Ore Control software I described above, internal to Mintec and within the mining industry. I saw things from the client side where Larry was the vendor and I represented the mining company. I can attest to the blood, sweat, and tears involved. Spending weeks away from your family doing an implementation in a decidedly unergonomic, dusty mine office and debugging software over the phone on Christmas Day are things you can't pay someone enough to do. Larry was a true believer in his product and did what was necessary to make it happen.

Ironically, Larry was promoted to Project Manager for MineSight Operations about eight months before his death. He had arrived after all that hard work. I would tease him about spending his time in a corner office with the head of the company smoking cigars and sipping scotch. He laughed. The guy never stopped working. He was very much a type A personality. The corporate life would have been easier on him and his family. I don't know how well he would have liked it long term. He loved to be in the thick of things, fixing things, making things work beautifully.

The main lesson I can glean from Larry's life is that if you want to do something significant, liking what you do is not enough. You have to care intensely and passionately about what you're doing. He was one hell of an individual.

Thursday, December 13, 2012

Plan for Pycon 2013

I looked at the Pycon talks and events schedule and was a bit overwhelmed.  There's just too much awesome, which is, for me, a bad thing.  Basically I've got to narrow down a strategy and a scope for attending this thing or I'll just learn nothing about everything and come back to work in Arizona and Africa totally burned out from lack of sleep.

What I've tried to do is group things in categories that are relevant to what I do for a living or what interests me to the point I would make the time to actually work on them.  I then listed the talks in these categories (mine, not the conference organizers') on paper and ranked them within each category.
The pattern that fell out for me is that I would be best served by focusing on Python 3.3 and how to use it best.  For me, that's going to be the theme of the conference, even if there are other more prominent official themes.

Disclaimer:  I'm a geologist/wannabe mining engineer/sometimes programmer/sometimes developer who is trying to write decent code and maintainable software.  Among the Pycon crowd, I'm probably well into the lower 50th or 25th percentile in terms of computer literacy and programming skills.  Among mining professionals I'm probably somewhere in the upper 25th percentile in terms of those skills.  It's all relative.  What's good for me is probably not the same as what's good for you.  Hopefully this gives other people some idea of how they would like to plan their conference experience.

A) Core Language for the User

1) How to Except When You're Excepting - Nam
2) Become a logging expert in 30 minutes - G. Roy
3) Iteration & Generators: the Python Way - Ramalho
4) Loop like a native: while, for, iterators, generators - Batchelder
5) Python's Class Development Toolkit - Hettinger
6) Transforming Code into Beautiful, Idiomatic Python - Hettinger
7) Encapsulation with descriptors - Ramalho

This is all basic language stuff.  I need the repetition and reinforcement.  "Yeah, but Carl, TWO talks on iteration?!  Don't you think you're taking this whole repetition/looping thing too far?"  No.  When you do summing of tonnes and grade and weighted averages and binning of material types and truck counts and processing of millions of individual 3D geologic model blocks for a living, there are never enough ways or efficient enough programming language idioms.  iter(), like, um, forever . . .

B) Python Internals

1) The Guts of Unicode in Python - Peterson
2) All-Singing All-Dancing Python Bytecode - Hastings
3) How Import Works - Canon
4) sys._current_frames(): Take real-time x-rays of your software for fun and performance - Rochael

I'm probably going to actually understand about 25% of what I hear IF I actually do some research ahead of time.  Still, knowing how something actually works can be a huge advantage.

C) Miscellaneous Stuff

1) Why you should use Python 3 for text processing - Mertz
2) How the Internet works - McKellar
3) Internationalization and Localization Done Right - Varshney
4) Python for Robotics and Hardware Control - Foote
5) Realtime Tracking and Mapping of Geographic Objects using Python - Burhum
6) Location, Location, Location - Grace
7) The Magic of Metaprogramming - Rush
8) Integrating Jython with Java - Baker
9) Getting Python and Django Through Your Java Shop Front Door (with Jython) - Wierzbicki

All stuff that I've had to deal with in the workplace.  All stuff that I wished I had been smarter about :-\

D) Community

1) Scaling community diversity outreach - Laroia
2) Designers + Developers: Collaborating on your Python project - Elman
3) How (Not) To Build An OSS Community - Lindsley

People hacking is not my strong suit.  Anna Ravenscroft gave a talk at Pycon back in 2010 on diversity.  I had a couple "Huh, I never thought of that.  Yeah," moments during that talk.  If I can glean a handful more of those "Huh" moments from these talks, they are more than worth it.

E) Tutorials

1) A Gentle Introduction to Computer Vision - Scott
2) Python 3 Metaprogramming - Beazley

We use a fair bit of computer vision in mining and that will probably increase as we move into the future.  The main application I can think of is estimation of rock fragmentation size with SPLIT imagery.  I actually own a VB book that covers a lot of basic concepts in a programming context (Visual Basic Graphics Programming - Stephens).  The problem is that it's 1) in a dead language - VB6 and 2) not in Python.

Beazley's course is going to be sick.  I will be truly disappointed if my head doesn't explode along with everyone else's and there's not a mini zombie apocalypse inside the classroom ;-)

F) PyPgDay 2013 - San Francisco PostgreSQL User Group

I've been to these PostgreSQL days the day before OSCON a time or two.  They are generally fantastic.  I really like the PostgreSQL community and the rigor they put into making robust database software.  Berkus or Deckelman, or both should be there.  Good, smart folks!  I am looking forward to this.

F) Keynotes

1) Hettinger
2) McKellar

Confession:  I am not into keynotes.  Basically it's first thing in the morning, you're beat from travel.  You herd into a huge dark conference room with a zillion other cattle, er, people.  If the talk, like a movie, doesn't make you forget about how you feel it can be a bit of a soul crushing letdown.  Back in 2010 Guido changed the format of his keynote to a twitter Q & A.  I hated it; some people loved it.  Yes, I am an uptight inflexible old fart who could stand to loosen up.  Yes, Guido is an indisputably great guy.  That said, I don't think I'll re-Guido myself on the keynote end for some time to come.

Rasberry Pi is a wildly cool innovation.  Sadly, I still really haven't successfully hacked on the Arduino.  When people are mastering the great grandchild to the Rasberry Pi, I'll finally be getting started on the Arduino when I'm in a nursing home.  Same problem as I mentioned at the outset - too much awesomeness, too much of a good thing.

Hettinger is really in the zone on the Python language end of things and cool new ways to use it.  He tweets tips constantly and has an infectious enthusiasm for the language.  I don't think he's reached his ceiling with respect to innovation or burn out.  Safe bet.

McKellar is a Linux kernel engineer, if I understood her bio correctly.  So here's this woman who has really made it in the industry and isn't wanting for anything accomplishment-wise or employment related.  Nonetheless, she is taking the time to get involved in outreach.  To me that says passion.  I'd like to hear what she has to say, whether she's talking about kernel hacking or outreach.

G) Conference Prep

1) Install or build Python 3.3 and dink with it.
2) Get a real database with real world data that mean something to me set up on Postgresql on my OpenBSD laptop.
3) Brush up on Unicode and Python's Unicode capabilities in preparation for Peterson's talk.

That's about it.  See you at the con.