Avoiding a data hotpot this winter

Data Hotpot

As the weather gets colder, you often find savory hotpots dished up, with all kinds of ingredients. But in data analytics it can be better not to put too many ingredients in a single pot for analysis. Or to know exactly which ingredients are right to enjoy a clear structure — or even to deconstruct, as you might say.

In an earlier blog post we looked at the subject of “incomplete orders”. Sometimes the question arises of which order documents from SAP® you should at all include in data analysis, and which you should exclude. Here you should find some useful suggestions.

The data basis we proceed from here focuses on the tables EKKO (order document headers) and EKPO (order document items); screenshots use the desktop variant of ACL™ Analytics software.

Orders from SAP® – the basics

Briefly to the basics: You can have orders shown by transaction ME23n. They are also divided into header information and item information. If you do not particularly care for data models, you can imagine it as a paper-bound order: The letter header contains information applying to the entire order, e.g. to which supplier it is sent, for when it is dated, but also what terms of payment apply, and in what currency the order is to be understood. Item information refers to the single articles, and can be different from one item to the next, like number (10 laptops vs. 8 docking stations), and the different price per item. Header information for orders is in the EKKO table, item information in the EKPO table. The common criterion joining the tables is the purchasing document number EBELN.

How to selectively analyse your order data

Once you have these order transactions in your analytics software, the following aspects can be important for you to narrow down the data basis:

1. Only what it says is in it
2. Do not ignore what is deleted
3. Shut out certain internal activities

Now for a look at the details of these three points:

1. Only what it says is in it

An important point is that although the SAP® tables named — EKKO and EKPO — are called “purchasing document header” and “purchasing document item”, they also contain other types of purchasing document. So you also find requests for quotations, contracts (outline agreements), or scheduling agreements in these tables. The reason is that, technically seen, they are of the same data structure as the actual orders — although this is actually a kind of “metadata”, in part not really used until there are release orders. The pure existence of a contract in the SAP® system for € 1,000,000, for instance, does not yet say anything about its actual use. From the viewpoint of the data analyst you can differentiate this quite simply by looking at the BSTYP field in the EKKO table. In the following the field was condensed with ACL™ Analytics (or the BSTYP field, which only contains the acronym, and its label to make it better legible).

Figure 1 – Document types in purchasing

The named characteristics are quite clear. Of 16,085 order documents overall, only 15,882 are actual order documents, the remainder splitting into 73 contracts, 122 scheduling agreements, and 8 requests for quotations.

To avoid errors in data analysis (e.g. it would be very misleading to present values for contracts and orders and requests undifferentiated in a single total) it makes sense to differentiate analysis by document type, or to exclude those that are not relevant right from the start.

2. Do not ignore what is deleted

There are many examples in which data records deleted in data analysis, or data records identified as deleted can be ignored. Proper deletion occurs in the rarest cases because of the obligation to retain or record. Instead, data (both master data and transactions) exhibit an attribute often called a “delete flag”. If an “X” is entered for instance, the document or master data are marked as deleted. This is how the term “deleted” is used in this article too. If I write “deleted” this means data records with an appropriate delete flag, but that may easily still be found in the tables. This is less likely to occur in financial transactions because there is an offsetting entry or cancellation. More frequently however in advance processes like purchasing or sales, or the associated master data. In the latter case it can sometimes really make sense to shut out data records marked as deleted: If you analyse duplicates in vendor master records, for example, to avoid double payment, or enhance data quality, deleted master records should be excluded from analysis because these cannot be used per se in bookings. A division would not react positively if the list with potential vendor duplicates also contained master records ready labeled for deletion.

The situation is different with purchasing documents. Here you find a possible delete flag both at header level (table EKKO) and at item level (table EKPO). As a rule, single items can be marked as deleted, but the whole document can only be labeled for deletion if all items in it are deleted.

In the purchasing area however, it should be noted that deleted orders or order lines should not absolutely be excluded from analysis. The reason is that often, from the viewpoint of ordering operation, orders or order lines are marked as deleted if they are seen to be finished. That should prevent their further use, although the activity is finished in content (even if there are still quantities remaining in an order that were not yet called for and released).

But that in turn means that there may quite easily be goods receipts and invoice receipts for an order item that are interesting for analysis. If you excluded orders or order items marked as deleted, you would entirely miss out these operations, and the resulting picture of quantity and value structure of ordering operations would be a false one.

3. Shut out certain internal activities

A final point I want to point to is what are called stock transport orders. These are usually of document type UB, and serve for internal stock movements or transport of material between plants.

Stock transport order
Figure 2 – Stock transport order

They are easily shut out by the document type (field BSART = “UB”), and are also identified in that the vendor number (field LIFNR in table EKKO) is not filled in. To avoid confusion, e.g. when totaling by vendor, or to form distinct order totals, these could be shut out of analysis or looked at separately.


I hope the blog post has given you something of an impression of what to look out for when analysing orders from SAP® so that the data basis for your analysis is as clear and well defined as possible, and your data analysis produces informative results.

Monday, December 4, 2017 In: Hot Topics, Newsletters Comments (None)

Contact us

3 Appleton Court, Calder Park
Wakefield, WF2 7AR

+44 (0) 1924 254 101


Mailing List

Subscribe to our newsletter.