An Exercise In Optimization

Sometimes when developing code you can spend to much time trying to optimize code that does not need to be optimized, especially if it makes the code harder to understand (often negatively referred to as "premature optimization").  However, sometimes optimization can be very worthwhile.

A quick example to share.  I was working on a segment of code last week that was taking a long time to run; on the order of 5+ minutes for something that seemed like it should be much quicker.  The code made sense when looked at, but it definitely felt as though this one might be worth spending time to optimize.

Upon closer inspection, I realized that a key index was missing from the database lookup.  This code has recently changed from processing a flat file to pulling information from a Sqlite database.  The table it was having to query was large, and the column it was querying against wasn't part of an index.  Upon adding the index, the run time of the code dropped from over 5 minutes down to 31 seconds.

All done!  Or not.  Once you are there, it's worth making sure you are not overlooking other possible issues.  I realized it was also querying one other table that, although it was smaller, was suffering from the same issue.  After adding an appropriate index for that table as well, the run time had dropped to 15 seconds.

But that's not all!  Upon a closer review of the code, I realized it was repeatedly looking up data with the same parameters in a lot of cases, so I added an in memory cache to cache that data as needed throughout the processing.  This dropped the total run time down to 5 seconds.

And then I realized I had also left out from the caching another lookup that was only needed when populating the cache.  After moving that last lookup to the code that utilized the cache, the run time had dropped again down to 14 milliseconds.

From 5+ minutes to 14 milliseconds.

I wasn't aware I could make it that fast, but by being mindful of what seems reasonable and being careful in my review of how the processing was operating (and not exclusively looking at just the database or just one segment of code), I was able to dramatically improve it.  Sometimes optimization can be very worthwhile.  Just use your best judgement on where it seems worthwhile, and where it would just complicate the code.

Lost in a Wormhole

The blackness of space ravages my mind.

It has been almost a year since that fateful day. I had found new wealth in shipping goods between regions -- my small contribution to serving the people of the Empire. I finally had the resources to buy the finer ships of the galaxy, including the largest freighters to support my trading federation. But I wasn't content. Null space was a mess, overrun by savages lacking any code of conduct. High security space was a bore. And with pirates as active as ever in high security shipping lanes, my freighters were locked up in space port with most of my shipments still using the sexy but limited deep space transports.

That is when I saw the shimmer. It drew me in a first glance. It drew me in with a dance of colors that ebbed and flowed that seemed somehow both random and orderly at the same time. It was like no wormhole I had encountered before. On any other day I might have turned away to avoid the unseen dangers, but this was no ordinary day. Dissatisfied with known space and mesmerized by the wormhole to worlds unknown, I charged forth with all the bluster of a drunk sailor on a mission.

That was almost a year ago. I've seen amazing worlds and unbelievable stellar phenomena, but I am lost among the stars. I yearn to go home without any idea how to get there. At my wits end, coasting on the fumes of a starving fusion reactor, I suddely saw something familiar. A glimmer of light. And a glimmer of hope. First, a moment of disbelief. Then, of heighened excitement, as I burst from my chair and spilt my drink. The way home! That's it! Then, finally, a feeling of dread. What husk of a shell of what was already a decaying Empire would I find when I get back? Would anything be the same? Would anything be left worth returning home for?

My sudden dread came about so suddenly as to leave me in shock. At that moment, I remembered the Mystic who spoke to me many years ago. Such an odd encounter at the time. "Heart and mind", she told me. Alone, the heart can leave you in chaos, and the mind can make you cold. Only together can they find balance, and lead you out of the dark. "You will find yourself in the dark one day", she said. "Let your heart be the Captain, and your mind the Navigator."

I think I understand now. My heart yearns to reconnect with my people. To try to make things right. My mind can help me get there, and keep me steady on the journey. It's time. I'm going home.