How to Optimize SQL SELECT Statement When Retrieving Data from Invoice History

How to Optimize SQL SELECT Statement When Retrieving Data from Invoice History

'Release Date:11/29/2017

Q - I used the following SQL statement to retrieve invoice history data for a certain item within a certain date range.  It took forever to run.  Please help.

SELECT CPINVLIN.INV_ITM_ITM_NO, CPINVLIN.INV_ITM_INV_DATE, CPINVLIN.INV_ITM_DESC_1,
CPINVLIN.INV_ITM_REASON_CODE, CPINVLIN.INV_ITM_DESC_2, CPINVLIN.INV_ITM_QTY_ORDER,
CPINVLIN.INV_ITM_TOT_QTY_SHP, CPINVLIN.INV_ITM_UNIT_PRICE, CPINVLIN.INV_ITM_UNIT_COST,
(INV_ITM_UNIT_PRICE*INV_ITM_QTY_ORDER) AS ExtPrice, CPINVLIN.INV_ITM_INV_NO,
CPINVLIN.INV_ITM_PROD_CATE, CPINVHDR.INV_TYPE, CPINVLIN.INV_ITM_CUST_NO,
CPINVHDR.INV_SHIP_TO_NO, CPINVHDR.INV_SHIP_TO_NAME, CPINVHDR.INV_SHIP_TO_ADDR_1,
CPINVHDR.INV_SHIP_TO_CITY, CPINVHDR.INV_SHIP_TO_ST, CPINVHDR.INV_SHIP_TO_ZIPCD,
CPINVHDR.INV_SHIP_TO_COUNTRY
FROM CPINVHDR INNER JOIN CPINVLIN ON CPINVHDR.INV_NO = CPINVLIN.INV_ITM_INV_NO
WHERE CPINVLIN.INV_ITM_ITM_NO='RU1022-22'
AND CPINVLIN.INV_ITM_INV_DATE>=20160101
And CPINVLIN.INV_ITM_INV_DATE<=20171130;

A - I used the same SELECT statement against my test database where CPINVHDR.BTR is 750MB and CPINVLIN.BTR is 540MB. These two tables initially are not cached in the server memory, so it took 15 minutes before it returned the data. Then I tried it second time. Since the data was already cached in server memory, this time around it took 26 seconds. Even though the second time is significantly faster, I still don’t like the fact that it took 26 seconds. So I tried to see what I could change to make it faster. The following revised SQL SELECT statement took only 2 seconds:

SELECT CPINVLIN.INV_ITM_ITM_NO, CPINVLIN.INV_ITM_INV_DATE, CPINVLIN.INV_ITM_DESC_1,
CPINVLIN.INV_ITM_REASON_CODE, CPINVLIN.INV_ITM_DESC_2, CPINVLIN.INV_ITM_QTY_ORDER,
CPINVLIN.INV_ITM_TOT_QTY_SHP, CPINVLIN.INV_ITM_UNIT_PRICE, CPINVLIN.INV_ITM_UNIT_COST,
(INV_ITM_UNIT_PRICE*INV_ITM_QTY_ORDER) AS ExtPrice, CPINVLIN.INV_ITM_INV_NO,
CPINVLIN.INV_ITM_PROD_CATE, CPINVHDR.INV_TYPE, CPINVLIN.INV_ITM_CUST_NO,
CPINVHDR.INV_SHIP_TO_NO, CPINVHDR.INV_SHIP_TO_NAME, CPINVHDR.INV_SHIP_TO_ADDR_1,
CPINVHDR.INV_SHIP_TO_CITY, CPINVHDR.INV_SHIP_TO_ST, CPINVHDR.INV_SHIP_TO_ZIPCD,
CPINVHDR.INV_SHIP_TO_COUNTRY
FROM CPINVLIN, CPINVHDR 
WHERE CPINVLIN.INV_ITM_ITM_NO='RU1022-22'
AND CPINVLIN.INV_ITM_INV_DATE>=20160101
And CPINVLIN.INV_ITM_INV_DATE<=20171130
AND CPINVHDR.INV_NO = CPINVLIN.INV_ITM_INV_NO;

So why is the second SQL statement significantly faster? Essentially, I took the inner join out of the “FROM” clause. Instead, I implied the inner join in the last WHERE condition. Why does this help? This is because in the second SELECT statement the WHERE clause is constructed so that PSQL will use the first condition to narrow down CPINVLIN records to that item number = ‘RU1022-22’ first. INV_ITM_ITM_NO is a key field of the CPINVLIN table. So this is done quickly and produced a small data set. Then it further narrowed down the number of records with the line item invoice date range. Finally, with a small data set, PSQL performed the join from CPINVLIN to CPINVHDR in the last WHERE condition.

On the other hand, the first SQL statement caused PSQL to perform an inner join of the entire two database tables CPINVHDR and CPINVLIN first. Both tables are big, so this is why it was slow. Finally, when these two tables were joined, the system narrowed down the records with the WHERE condition. This is why the first statement was so slow.

Unfortunately, PSQL does not always know whether it should perform the join first or the where condition first to optimize performance. So we have to evaluate each scenario to influence PSQL to make the right choice in order to run faster. Most Elliott users do not have the expertise to do this kind of SELECT statement optimization, so don’t feel bad if you don't understand this article. This is what Netcellent does best, so just talk to us and we will help you.


EMK


    • Related Articles

    • How to Optimize SQL SELECT Statement for BOMP Product Structure

      Release Date: 11/29/2017 Q - I used the following SQL SELECT statement to retrieve product structure from the BOMP module. Generally speaking, it works. But the performance is not what I would like it to be. Is there anyway to make it run faster? ...

    • How to Improve Query Performance When Retrieving Data from Notes & Invoice History

      Release Date: 12/12/17 Q - I am trying to get all SHIP notes with related invoice information for a specific customer for a specific day. The speed of my attempts has been okay, but it feels like it could perform much faster. How can the below be ...

    • How to Retrieve Open Purchase Orders Through SQL SELECT Statement

      Released Date: 9/11/2017 Q - How do I retrieve an Elliott purchase order from a third party system through ODBC or ADO.NET? A - If you only need the purchase order header, you can use the following SELECT statement: SELECT * FROM POORDHDR WHERE ...

    • Feature - Invoice History Archive

      Release Date: 8/23/21 Version: 8.6 and Above The current invoice history data tables CPINVHDR, CPINVLIN, CPINVOPT, CPINVMTL, CPINVRTG, CPBOXFHS (history version of CPBOXFIL), CPBOXHSR (history version of CPBOXSER), CPBOXHST (history version of ...

    • CP1700 Customer Order Processing Invoice History Inquiry

      Invoice History Inquiry Application Overview This function gives you fast and easy access to all the necessary information to provide customer service for posted invoices on file for a customer. It allows you to quickly access and display a customer ...