May 14, 2014
When you’re building a server product that will support C++ APIs you need to consider your ABI – the binary interface. Typically, C++ APIs are distributed as headers and libs. If the functions your API exports include parameters that use, for instance, std::string, you immediately have a problem, as you’re requiring client code to use the same STL implementation as you did to build the lib. That’s OK if client code has access to the source, and can rebuild. But commercial, proprietary products, tend not to distribute source. So how to avoid forcing dependencies on API client code? I went searching for some resources, and found two especially good ones I had to flag.
Here’s Thiago Macieira on binary compatibility: an excellent presentation with guidelines for library authors. Here’s a summary of Thiago’s recommendations…
- Use pimpl idiom to hide object size
- Use plain old data types in function signatures
- Don’t hand out ptrs or refs to internals
- No inline funcs
- All classes need one non inline virtual; probably the dtor
- Avoid virtuals in template classes
- Do not reorder or remove public members, or change access levels
2 means no STL or Boost types in function parameters. I’ll address 6 by avoiding templates in my API.
This article by Agner Fog is a superb detailed survey of data sizes & alignments, stack alignment, call conventions for register usage and parameter handling, name mangling schemes and object file formats inc COFF for all the major x86 & x64 OSes and C++ compilers. Strongly recommended.
May 10, 2014
I’ve been getting into windbg while working on my POC. I’ve long been a fan of Microsoft’s Visual Studio debugger. Even back in the late 90s, when I got a serious case of Open Source Religion after falling under the spell of Eric Raymond’s Cathedral and the Bazaar, and went through an anti Microsoft phase, I never stopped rating the debugger. Visual Studio debugger is great, but windbg is a whole ‘nother thing. It’s the debugger MS use themselves for debugging Windows. Yes, the interface is a little clunky compared to the VS debugger, but the power of the command set more than compensates. It’s got it’s own scripting system built in, so you can construct custom breakpoints: for instance, break the 5th time round the loop when this int is greater than 100. And it has an API too, so the debug engine can be driven from other languages like Python. Personally, I’m really enjoying discovering the power of windbg. While I do so I’m capturing tips and tricks here.
May 5, 2014
Tales flags some interesting developments in the Python world. The demand for Python developers in finance does seem to be building. Both Man and Getco are big users, and as Tales points out, JP Morgan and BAML both use Python as the primary programming languages in their Athena and Quartz systems, both of which are inspired by Goldman’s SecDB/Slang. Tales wonders if Washington Square Tech will the fourth implementation of this paradigm; I believe it may be the fifth, as Morgan Stanley had an Athena like project called Pioneer during Jay Dweck’s ill fated tenure. Apparently that project is now defunct. The Athena paradigm is technically a very powerful solution for trading businesses that have run on ad hoc solutions using Excel for pricing and risk. Partly because they seek to replace Excel based pricing and risk, and partly because it’s compute efficient, all implementations of the Athena SecDB/Slang paradigm implement Directed Acyclic Graphs. I’m guessing that Washington Square Tech will think this could be very appealing for buy side firms that don’t have big in house tech stacks, together with incumbent tech teams defending them against replacements.
Directed Acyclic Graphs (DAGs) are a powerful implementation technique for minimising the load of compute intensive tasks like pricing and risk, and this is one good reason why Excel has been so successful in this area, as Ben Lerner of DataNitro explains persuasively here. So it should be no surprise to see that Man Groups own open source Python codebases include a DAG implementation, MDF. The MDF docs include a very good illustration of the power of the DAG approach.