DIY Betting Bots
Betfair Software
Wednesday, 28 May 2008
I've just noticed an annoying little change Betfair made to their site in last night's software upgrade. The "What If" feature (that tells you the financial impact of a bet before you make it) no longer works for bets below the £2 minimum. So now when some annoying toerag matches £1 of one of my bot's bet, and I want to green up with my own £1 bet, I'll have to crack out the calculator and do the sums myself. It's almost as if Betfair don't want me to break their rules on minimum bet sizes!
Logging in to Betfair with ZSI
Friday, 23 May 2008
This week I've been working out how to access the Betfair betting exchange's SOAP API using the Python ZSI library. This API allows an automated program (or 'bot') to retrieve details of the available odds, and place and monitor bets.
Creating the Client Code
ZSI includes a script wsdl2py that automatically generates client code for accessing a SOAP web service from the WSDL file defining it.
For regulatory reasons, the Betfair service is split across two different exchanges (one in the UK and one in Australia) and has three WSDL files. One WSDL file describes global services (applying across both servers) while the other two describe the exchange-specific services.
The commands to generate client-side Python code for the global and UK-specific WSDL files are as follows:
wsdl2py -bu https://api.betfair.com/global/v3/BFGlobalService.wsdl wsdl2py -bu https://api.betfair.com/exchange/v5/BFExchangeService.wsdl
Running these commands generates Python modules BFExchangeService_services.py and BFGlobalService_services.py containing Python code to access the services.
Associated helper modules BFExchangeService_services_types.py and BFGlobalService_services_types.py define Python classes corresponding to the various complex types in the WSDL file.
Calling the Betfair Login Service
As our first example of using the client code to access a SOAP service, let's use the login method, which is defined in the global WSDL file. The first step is to import the module produced by wsdl2py, and create an object for accessing the services defined in that file:
from BFGlobalService_services import * BFGlobal=BFGlobalServiceLocator().getBFGlobalService()
Now we need to construct an object with the input parameters for the login service. The client code generated by wsdl2py defines a class servicenameIn to hold these parameters i.e. loginIn in this case. The WSDL file reveals that the service has a single parameter called request with complex type LoginReq:
<xsd:element name="login">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="request" type="types:LoginReq"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
So the next bit of Python code we need looks like this:
req=loginIn() # object to hold input parameters for class login loginrequest=req.new_request() # object to hold complex input parameter 'request'
Note that new_paramname is a factory method created by wsdl2py for the complex parameter named paramname. So now we have a Python object corresponding to the complex datatype LoginReq. To find out what attributes this has, we can look at the XML again:
<xsd:complexType name="LoginReq">
<xsd:sequence>
<xsd:element name="ipAddress" nillable="false" type="xsd:string"/>
<xsd:element name="locationId" nillable="false" type="xsd:int"/>
<xsd:element name="password" nillable="false" type="xsd:string"/>
<xsd:element name="productId" nillable="false" type="xsd:int"/>
<xsd:element name="username" nillable="false" type="xsd:string"/>
<xsd:element name="vendorSoftwareId" nillable="false" type="xsd:int"/>
</xsd:sequence>
</xsd:complexType>
Because the attributes of the LoginReq are all simple types (ints and strings) the Python code to populate them is straightforward. Note that 82 is the ProductId for the free API:
loginrequest.LocationId=0 loginrequest.VendorSoftwareId=0 loginrequest.ProductId=82 loginrequest.IpAddress='' loginrequest.Username='username' loginrequest.Password='password'
Finally, having created our loginrequest object, we can use it to call the SOAP service:
req._request=loginrequest response=BFGlobal.login(req)
Interpreting the Response
Having called the method, and stored the response, what do we do with it? The WSDL file defines the LoginResp class as follows:
<xsd:complexType name="LoginResp">
<xsd:complexContent>
<xsd:extension base="types:APIResponse">
<xsd:sequence>
<xsd:element name="currency" nillable="true" type="xsd:string"/>
<xsd:element name="errorCode" type="types:LoginErrorEnum"/>
<xsd:element name="minorErrorCode" nillable="true" type="xsd:string"/>
<xsd:element name="validUntil" type="xsd:dateTime"/>
</xsd:sequence>
</xsd:extension>
</xsd:complexContent>
</xsd:complexType>
<xsd:complexType abstract="true" name="APIResponse">
<xsd:sequence>
<xsd:element name="header" nillable="true" type="types:APIResponseHeader"/>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="APIResponseHeader">
<xsd:sequence>
<xsd:element name="errorCode" type="types:APIErrorEnum"/>
<xsd:element name="minorErrorCode" nillable="true" type="xsd:string"/>
<xsd:element name="sessionToken" nillable="true" type="xsd:string"/>
<xsd:element name="timestamp" type="xsd:dateTime"/>
</xsd:sequence>
</xsd:complexType>
Sure enough, our response object has a Result attribute, whose own attributes correspond directly to the dataitems in the XML above (except for being capitalised). The most significant ones for our purposes are:
- response.Result.ErrorCode is a code indicating whether the login was successful. A value of 'OK' indicates that it was, while any other value (such as 'INVALID_USERNAME_OR_PASSWORD') indicates that it was not.
- response.Result.Header.ErrorCode is another error code, that can contain generic errors not specific to the login service. Your code should check that both Errorcode and Header.ErrorCode are equal to 'OK' to confirm that the login is valid.
- response.Result.Header.SessionToken is a string that identifies an API session. It should be stored, as it will be needed to make subsequent API calls.
Pulling it All Together
The sample source code (see right of this page) pulls all of the above explanation together into a simple bflogin function that calls the login service, returning True if it succeeds, and storing the sessionToken for subsequent use.
Building a SOAP Client in Python
Wednesday, 21 May 2008
I've spent some time the last couple of evenings trying to work out how to access the Betfair API from Python. Having heard so much about Python's "batteries included" philosophy, it was a bit disappointing to find that there's no SOAP support as standard (unlike PHP5, which I used to build my first bot).
My starting point was SOAPpy, as described in Dive into Python. But it soon became clear this wasn't a good choice - it doesn't seem to be supported any more, it relies on other unsupported libraries, and (perhaps more importantly) I couldn't get it to work with the complex data types used by the Betfair API. So I decided to try the Zolera Soap Infrastructure (ZSI), which appears to be a more current alternative.
Installation
Installation was slightly daunting for a Python newbie such as myself:
- ZSI apparently uses an (unsupported) XML-processing library called pyXML. There's been no new version of it since 2004, and there isn't an official binary downloadable for Python 2.5 (which is what I'm using). I did however find a blog with a link to a pre-built Windows binary, which was nice. Clicking on the executable installs.
- ZSI itself is distributed as an egg file, which is apparently a package format used by a Python package manager called Easy Install. I found and followed instructions on how to install Easy Install on a blog here.
- I was then in a position to download the ZSI software (ZSI-2.0-py2.5.egg), remove the spurious .zip extension Windows XP added to the filename, and install it by typing easy_install ZSI-2.0-py2.5.egg.
Documentation
The official documentation seems pretty daunting, but a little bit of googling around reveals some more novice-friendly information:
- There's a User's Guide which (although still quite technical) does at least focus on how you might use ZSI to generate a SOAP client, rather than diving into the internals.
- Chris Hobbs has written a cookbook with some helpful-looking examples
- The European Bioinformatics Institute have some examples of how to access their own SOAP APIs using ZSI
So, armed with those, let battle begin!
Oh Dear - I Seem to be Rubbish at Betting!
Saturday, 17 May 2008
Ran some reports tonight on all my betting so far this year - I've got a program that downloads all my bets from Betfair into a mySQL database - and was disappointed to find that I've not actually made any money. My old PHP5 bot has been slogging away, making a pound here and a pound there, but I've lost almost all of it with unwise political bets (on Ken Livingstone and Hilary Clinton mainly). So overall I'm only a tenner up for the whole 4½ months.
The lesson seems clear: I should stick to the automated algorithms, and resist the temptation to take random punts on things I don't really understand!
Plans for a New Bot
Thursday, 8 May 2008
I started work today on designing my new betting bot. It's going to be written in Python, and will:
- Wrap a generic interface around the API of each betting exchange, so that the same code can trade on Betfair, Betdaq and other exchanges.
- Collect and store price data for any number of markets.
- Provide a general framework that a variety of different betting and trading algorithms can plug into.
It has to be said that I'm probably not the best qualified person to do this. I'm a relative newbie at Python, and at betting too - started less than a year ago, and made a few hundred pounds so far. I do have some experience with the betfair API, as I've got a crude PHP bot that consistently makes (very small amounts of) money. But the new one is intended to be a lot more flexible.
First big task - write some Python code to trawl back through the historical price data I've accumulated, and try out various indicators for predicting which odds are likely to shorten, and which to drift.