Re: Efficiency of XForm xform_run method

From: mda
Keywords: XFormDBF, VFP
Remote Name: 24.147.233.252
Remote User:
Date: 09/07/00
Time: 14:51

Comments

Thanks for the suggestion.  I'll keep your idea in mind next time I return to the task of investigating ways to further speed up large transformations (xfma0049).

The present behavior is not as inefficient as what you described, but I can understand how you were misled by the somewhat complicated logic, which is due to the residue of an earlier unoptimized design.  There are a few optimization-related properties and methods that pertain to this:

property xfop_compile:

Flag property indicating that use of VFP's COMPILE is enabled (the new default).  Older optimization logic uses &-macros.  This property is always set to .T. (via the property sheet), but it can be manually changed via the VFP Form Designer.  (There is no checkbox to control this property in the Options page.)  I used this during development to benchmark the effects of compiled vs non-compiled modes.  I didn't see any real need to make this a user-settable option, but I retained the old logic just in case it has some usefulness in the future.  Since SP3 gives us a runtime compile capability, there's no obvious reason to ever need the old un-compiled mode logic to which you referred.

property xfop_replfieldbyfield:

Flag property indicating that less efficient field-by-field replacement logic should be used.  This corresponds to the user-settable "Don't Optimize" checkbox on the Options page.  I made this option available because it could be helpful in debugging transformations.  When you run a single-record Test, this property is temporarily turned on, for better error diagnostics.  For the Run command, you would normally want this property turned off (the default setting), because it makes a huge difference in performance, especially for large, wide tables.

method xfom_run:

Method to run the full transformation and return a success flag.  This governs the outer loop over input table records, calling method xfom_run1 on each loop cycle.  For optimized modes, i.e. when not forced into field-by-field replacement, there is a single initial invocation of the xfom_buildrepl method.

method xfom_run1:

Method to run the transformation against the current input record.  This is used to support both the Test command button and the main Run command.  The detailed logic depends on the above property settings.  For the normal compiled and optimized mode, the pre-compiled program generated by xfom_buildrepl would be invoked here, which does not use any inefficient &-macro expansions.

method xfom_buildrepl:

In uncompiled mode, this method builds the array of REPLACE command line clauses that will be macro-expanded to transform each record.  However, in compiled mode this method generates and compiles a program to perform one or more REPLACE commands that process multiple fields at once, without &-macro usage.  Multiple REPLACE commands are generated only when the certain undocumented VFP limits are encountered.  These limits are determined by the properties xfop_replmaxfields and xfop_replmaxlen, which correspond to the maximum # of REPLACE fields and the maximimum allowable length of a single VFP REPLACE statement.  I rounded off the limits I determined experimentally to 128 fields and 8000 characters, so it's unlikely that more than 2 REPLACE commands per record would ever be required.

Brought to you by SpaceTime Systems.  All rights reserved.