Checking for and Repairing Improper Data in Elliott Database Files

Checking for and Repairing Improper Data in Elliott Database Files

Release Date: 05/20/2025
Versions: 8.6 and Higher

Background

From time to time, improper data will appear in the database files of Elliott.  For example, one or more null characters might be found in a date field.  Generally, these errors are caused by a programming error that went undetected during testing.  Most often, these errors will not cause any issues, but some newer technology that is being introduced into Elliott software will function better if these errors are eliminated.

Elliott has a database utility called DDF2BTR that is used to convert older database files to a newer, more flexible format (for example, to support alphabetic document numbers).  Using DDF2BTR in this way rebuilds database files in exclusive mode and cannot be done while others use Elliott.

With this release, we have added the ability to use DDF2BTR to check for improper data (null characters) and/or repair such data errors (replace nulls with spaces).  In this mode, database files are not opened exclusively, so others can use Elliott while you run the utility.  The following sections describe how this can be done.

Checking for Improper Data

Use the following command to check for improper characters in a range of files. 
For best results, log into the database server and go to the Bin86 directory in a command prompt and type a command like this:
      DDF2BTR /C M:\Elliott8\DATA\*.BTR
  1. /C tells the utility to check for and report improper characters.  Each file is opened in read-only mode and no data will be changed.
  2. In this case, the utility will work on all database files (*.BTR) in the DATA folder (for Company 01).
  3. The log produced by DDF2BTR will identify the improper characters it finds.
  4. This option can be run while other users are working in Elliott.
  5. It can be useful to specify subsets of the database rather than all files in one execution.  That will allow examining multiple files at the same time.

Repairing Improper Data

Use the following command to repair improper characters in a range of files. 
For best results, log into the database server and go to the Bin86 directory in a command prompt and type a command like this:
      DDF2BTR /R M:\Elliott8\DATA_02\AP*.BTR
  1. /R tells the utility to look for and repair improper characters.  Each file is opened in input/output mode.  Each record that has one or more improper characters will be repaired and written back to the database.
  2. In this case, the utility will work on all Accounts Payable database files (AP*.BTR) in the DATA_02 folder (for Company 02).
  3. The log produced by DDF2BTR will identify the improper characters it fixes.
  4. This option can be run while other users are doing things in Elliott.
  5. It can be useful to specify subsets of the database rather than all files in one execution.  That will allow repairing multiple files at the same time and skipping over groups of files that show no need for repair.

Example Log File

  1.  5/20/2025  8:57:11 am
  2. Repairing data: 5  files in J:\ELI75\DATA_97\N*.BTR
  3. Using DDFs in .\DDF40

  4. NOT2.BTR     skipped due to missing DDF entries
  5. NOTE.BTR          13,353 of     18,510 records repaired in       26.52 seconds
  6.   Data error in  NOTETYPE.BTR: 000000011000000011            : x"00"  fixed at  240
  7.   Data error in  NOTETYPE.BTR: 000000062eltonsmart@gmail.com : x"00"  fixed at  240
  8. NOTETYPE.BTR           2 of         11 records repaired in         .09 seconds
  9. NSCTLFIL.BTR skipped due to data checking excluded
  10. NWSMFORM.BTR           0 of      2,955 records repaired in        2.29 seconds

  11. 3 files processed
  12. 1 files excluded from data checking
  13. 1 files skipped due to missing DDF entries
  14. NOTE: Complete with warnings -- please review all message(s) above...

  15. Total time: 41.91 seconds....
  16.  5/20/2025  8:57:53 am
Please note the following lines:
  1. Line 6 summarizes the number of records repaired for NOTE.BTR.  The utility knows that there could be a very large number of the improper characters, so it is programmed to not show the individual errors.
  2. Lines 7 and 8 show the improper characters in NOTETYPE.BTR that were found and repaired.
    1. After the file name, the first 30 characters of the file are shown to help identify the particular record fixed.
    2. X"00" indicates the improper value found.  It was replaced with a space character.
    3. 240 indicates the position of the improper character in the record.
  3. Line 9 summarizes the number of repaired records for NOTETYPE.BTR.
  4. Line 10 indicates that NSCTLFIL.BTR was skipped because the utility is programmed to exclude this file from processing.

Exceptions

Generally speaking, the Elliott database should not contain null values.  But there are exceptions.  Therefore, not all files can be checked or repaired.  What follows are the exceptions and their limitations:
 File
 Limitation
 Rationale
DEFEREML
Repair, no log
Possible large number of known errors
DEFERWIN
Skip file 
Contains binary data
GLBALFIL
Skip file
Contains binary data
GLCMPFIL
Skip file
Contains binary data
NOTE
Repair, no log
Possible large number of known errors
NSCTLFIL
Skip file
Contains binary data like encrypted password, checksum value
PRINTTPL
Skip file
Contains binary data
PRINTWIN
Skip file
Contains binary data
REPTPASS
Skip file
Contains binary data
SYACTLOG
Repair, no log
Possible large number of known errors
SYCONTCT
Repair and log but skip certain columns
Contains binary  (encrypted) data
SYEVTACD
Repair, no log
Possible large number of known errors
SYEVTHST
Repair, no log
Possible large number of known errors
SYMSEMTP
Repair, no log
Possible large number of known errors
WSORDHDR
Skip file
Webservice file, contains binary data
WSORDLIN
Skip file
Webservice file, contains binary data
WSSECLOG
Skip file
Webservice file, contains binary data


JEG
DDF2BTR
    • Related Articles

    • An Example of DDF Files in DATA Folder that Can Confuse PSQL

      Release Date: 04/11/2024 Version: All Q - I tried to access Elliott Item master column ITEM_CASE_SIZE in Report Desk. In one of the Report Desk reports, I wanted to drag ITEM_CASE_SIZE to the report body. But I can't find that column under the ...
    • DDF Files in DATA Folder May Confuse PSQL

      Release Date: 04/27/2018 Q - We recently noticed a strange phenomenon where if I try to access Elliott column names in the ARCUSFIL, I find the Customer AR account is defined as CUS_AR_ACCT_NO and it is 24 digits. See sample screen below: I have an ...
    • A Comparison of Elliott 7.5, 8.2, 8.5, 8.6 and 8.6 V2 DDF Files

      Release Date: 11/15/2022 Last Updated: 06/25/2024 Version: 7.5 & Up DDF Files are used to define the PSQL database schema. Generally speaking, we distribute five different types of DDF files for different versions of Elliott releases: 7.5 DDF: These ...
    • Elliott 8.6 Database Convention & Report Desk/PowerSearch

      Release Date: 08/27/2021 Revision Date: 4/11/2022 Version: 8.6 Elliott 8.6 Database Naming Convention Elliott 8.6 now supports Report Desk and PowerSearch, features based on the PSQL relational database engine. Databases must be created for Report ...
    • An Example of DDF Files in DATA Folder Causing Report Desk or Power Search Issue

      Release Date: 03/14/2025 Version: 8.6 & Up Q - When I tried to print a UDR report (i.e., AP Distribution History Report) from company # 98, I got the following error message: Cannot find definition of table APDSTHST in DDFs. You probably are using ...