Extracting Big Data from SAP

Quantitative research or analytic projects typically incorporate three main areas:

• Data collection and preparation.
• Analysis and interpretation.
• Reporting and distribution.

A setback at any step can lead to missed deadlines, delayed projects or worse. We recently had a client whose analytic tests were in place and their reporting and dashboard tools were ready to go. The only thing holding things up was their data collection held on an SAP system. The extraction tool worked as designed on small data sets but their analytics ran into problems when scaling up to automated big data extraction.

Our initial advice was to upgrade their software or switch to another tool. Implementing new software processes at a large company however, can be complicated. With deadlines fast approaching management made the decision to work with the tool they had, ACL Direct Link 6.1.

Data extraction in ACL Direct Link can be done in three modes: Extract Now, Background, and Scheduled (only available in 7.5). Extract Now is used for small queries whereas Background and Scheduled are used for larger ones (greater than 488kb). After submitting large queries in Background or Scheduled mode the data needs to be retrieved, but can only be done once the data is ready. If a retrieve command is sent before the data is ready on the SAP side the analytic will fail.

This is where automating the process can get tricky.  Our solution was to make use of SAP table TBTCO and implement the process below:

• Submit all queries in Background mode.
• Capture and log the SAP job names from TBTCO.
• For each un-retrieved table:
  • Check TBTCO and see if the data is ready.
  • If ready, then retrieve data and log findings.
  • Move on to the next un-retrieved table.
• Wait X minutes and repeat the step above.
• Once we have all the tables continue with the analysis.

This allowed the company to automate the download and auto-retrieve big data from SAP using ACL Direct Link. Crucially, it supplied the necessary data for their automated analysis scripts, the results of which could be packaged up into dashboards for reporting and distribution to wider audiences within the company.

Do you need to extract data from SAP?
dab:Exporter automatically converts data for use in ACL Analytics Desktop, ACL Analytics Exchange, Microsoft SQL Server, CSV or GDPdU. Read more about your options for SAP data extraction below:

http://www.dataconsulting.co.uk/software/acl-direct-link/
http://www.dataconsulting.co.uk/software/dab-exporter/