Daniel Beunza posts on his NYC presentation on visualization. Visualization is one of my pet themes – I’ve posted my thoughts on why it hasn’t been widely applied in trading. I’ll add a couple more observations I’ve made while building and deploying a 3D visualization system on our trading floor, at the same time as addressing a couple of points Daniel makes…

  • Daniel: “Traders are practical and resist innovation”. Traders resist innovation for innovation’s sake. If they can see an opportunity to profit from innovation, they’ll be very keen on it.
  • Traders know markets far more intimately than developers of visualization software ever can. Consequently visualizations can end up showing traders stuff they “just know”: stuff that has already been internalised and assimilated by the trader.
  • Visualization development cycles can be slow, especially if you’re coding in C++, which can make it hard to hold the attention of traders.
  • Daniel: “I believe that innovation will happen”. I used to share this belief, but I’m not so sure now. Seeing startups fail, and trying it myself has persuaded me that something very special will be needed.

I wonder if Daniel has come across Brad Paley’s work ?


September 26, 2006

Sparklines is a visualisation technique I’ve been following for a while – so I read this with interest. And then something else on cmiles blog caught my eye: a link to Juice Analytics. They have a nice blog on Excel hacks, which mentions a toolkit that gives you sparklines in Excel. Not the same Juice as founded by Charles Ferguson who wrote High Stakes – probably the best book I’ve read on founding a software start up.

Playing with R

August 1, 2006

I’m doing some data analysis at the moment, and I’ve gone through some toolsets in search of a combination that will give me power, expressiveness and charting capability. The raw data comes out of our trading system’s RDB. I started off with SQL queries, but soon found the slowness of complex queries and the arcana of SQL was holding me back. So I exported the data as CSV and set to work with Excel. I made some more progress, but didn’t want to resort to VBA for the ad hoc slice and dice coding. So then I switched to processing my 300Mb CSV with Python. Progress was good for a couple of days, but then my script started to get hairier. The data extraction and cleansing logic was all mixed up with the analysis logic. And I still didn’t have any good charts. Back to the drawing board. I vaguely remembered folk on Victor Niederhoffer’s mailing list discussing R. I checked out the programming recommendations on the great man’s site. So I downloaded R, and after 30 minutes playing, I’m very impressed. Scripting, powerful maths, built in vector and matrix data types, and flexible charting all rolled together. I’ll be asking around on the floor to see if any of our traders are using R…

Matt posted recently on visualization. I worked in the oil industry in the 90s, and did some visualization work, and I've often wondered why visualization isn't used more often in finance. After all, it is used heavily in many other domains that deal with high volumes of numerical data. For instance, medicine, science and engineering, and oil and gas.

Here are some factors that I reckon prevent visualization adoption on the trading floor…

  • Real time data: the domains that adopt visualization are often analysing static data sets that are produced by scanners (medicine) or simulators (oil). Timeliness is critical in the markets. Traders need to keep up with a flow of real time data. Visualizing real time data presents quite specific coding challenges, since rendering a 3D scene is computationally expensive.
  • No obvious spatial metaphors: I suspect it's no accident that visualization has taken off in sectors that deal with real physical objects, like the human body, or an oil reservoir. The visual representation of such objects is straightforward. The question of "what 3D form do I give this" doesn't arise.
  • Precious screen real estate: walk around our trading floor and you'll see traders with 6 flat screens driven by two PCs. All that screen real state is consumed with Bloomberg terminals, pricing blotters, RFQ blotters, trade blotters and Excel spreadsheets. If a trader is going to give up some space to a new tool, it better deliver real value, and not just seem like a 'nice to have'.

However, the search for competitive edge is unceasing in the markets, and there's always someone willing to try out a new technique. Cantor have a viz offering for US fixed income markets. 4D Trading were a start up that had a viz product. They found it difficult to sell, and were eventually acquired by GLTrade.

There are some niches in trading that do lend themselves naturally to viz techniques. I've seen Excel charting used for 3D plots of volatility surfaces. Derman has a great explanation of the implied volatility surface. A 3D plot of a vol surface that's been pulled out of a database of yesterday's closes can be a very handy sanity check for a trader coding his own pricing model in a spreadsheet.