Sunday 18 September 2016

Using MS SQL Server 2016 developer edition SSIS Data Tools to scrape websites.

Using SQL Server 2016 developer edition SSIS Data Tools (BIDS) to scrape websites for results and prices. The good news is that the above developer edition is now free and can be download here.

Forget the express editions as data tools are not supported on those editions. SSIS project is similar to Visual Studio projects you use to develop windows desktop applications and a lot more. While the projects that you will create in SSIS are for data centric tasks. You can extract data from websites and then use them to update your trading database that supports your betting rules/decisions.

You can query your database outside your trading bot software that will give you more insight on your data. The potential benefit of using SSIS is that you can run it all day long scraping results and prices and automatically updates your decision support database which is independent from your bot software. Your bot software can now access up to date information outside of Betfair API's.

If only Betfair will supply the sports results in their API's, life can even be less difficult.

No comments:

Post a Comment