Today I chaired a conference on Big Data. "Big Data" is one of those slightly amorphous concepts that may be designed to persuade you to buy lots of expensive database software, or it may describe an approach where you handle vast amounts of unstructured data in innovative ways, or it may mean nothing much at all. Or it may mean you should bin all your previous work and rewrite your website in Haskell.
Most of the problems of big data (with a small B) have been around for ages, long before the British were decrypting Nazi Enigma traffic, or physicists were running Monte Carlo simulations, or people were figuring out actuarial tables. Some of the problems are fixed now we have better technology, some problems remain. That meant that at least half the presenters spent their time decrying the existence of Big Data, as opposed to just plain data, and that felt like there might be a few straw men being erected just to be knocked down again.
I announced the speakers, and brought them up, and signaled to them when their time was up. This didn’t strike me as the hardest gig I’ve ever done (no crowds of rowdy drunks, just a bunch of silent business people) but apparently I did it well. Does nobody else remember to tell speakers when their time is up?
There were some interesting insights for me, like the fact that I’ve only ever worked in companies which haven’t been stuck with enormous legacy systems that trap them into old designs. And that when I worked at Holiday Extras, I was lucky enough to be able to build a data warehouse that really did capture pretty much all the data we had, rather than preside over a series of silos. And that somebody should be able to make a lot of money building a better way to give traders market information, and that perhaps I should rewrite everything in Haskell.
There was also some vendor flim-flam, like the man who spent half an hour building a report for us on stage, and acted as though we should be impressed (because it’s 1995, and the only way to get reports built is for the mythical "man from the IT department" to do it for us) and there were one or two presentations that seemed either inept or totally inappropriate for the subject at hand. I began to wonder what they’re talking about at other Big Data conferences.
Still, all these thoughts evaporated when I left the conference hall, and walked slap bang into a glass wall, with a thunderous bang as I bounced off it. Fortunately the wall was completely ok, but I went reeling. And fortunately, there were witnesses to my lack of awareness. This was on the back of almost walking into a mirror at lunchtime: perhaps I’m blinded by all the data at hand.
I walked home, getting slightly lost, and though I hoped to get a consolatory ice cream on the way, took the wrong street back and failed there too. Perhaps I need to stop walking into things.