Soundex and Miracode

Sometimes data are entered incorrectly because the person entering data mishears a word (most often a name); for example, enters Awbrey instead of Aubrey. This is especially true for English language, which has complex spelling rules. There are algorithms may help you to find all words that are pronounced closely to a sample. Given an English word, such an algorithm calculates a code that represents how the word sounds. Take a look at the following sample of an algorithm called Soundex:

Sample phonetic matches for name “Aubrey”

Here A160 is the calculated code of Aubrey. You can see that similar names like Awbrey or Abra have the same code. Such algorithms are called phonetic or phonetic indexing algorithms; the word “index” reflects the fact that the algorithm takes a single word and calculates a phonetic code, which is same for all words that sound alike. (The latter is not always true because algorithms aren't perfect). The code can be used on its own for searching, sorting, grouping and so on. In FileMaker terms this means the calculated code can be stored and used for fast searching and making relationships:

2006.01.03.02.png

Other algorithms that can make a fuzzy comparison work differently: they take two words and compare their similarity. This isn't as convenient as having an index of a word; every time you need to search for words similar to the given one you must check each record in a table. In FileMaker terms this means the calculation will be unstored and slow.

Most known of phonetic indexing algorithms are: the Soundex family, Daitch-Motokotoff Soundex, NYSIIS, Caverphone, and Metaphone; I wish I could be able to write a custom function for each. For now let's start from the simplest and the most famous Soundex and Miracode.

Continue reading "Soundex and Miracode" »

Handling exceptions

Let’s consider the problem of how to handle exceptions? Assume we have already chosen to go beyond the “method” of simply ignoring them by placing a Set Error Capture[ On ] step at the beginning of a script. Shouldn't we then just write code like this?

2005.12.11.01.png

Well, I think that though the code above is correct, it still has a problem: it mixes the exception part with the main functionality. It's difficult to follow such a code. Imagine there's three error-checking conditions in a script and you're reading it after a while; you'll have to open three script steps and read their formulas to understand what's going on.

Though you cannot avoid the mix, it worth trying to clearly separate the part that handles exceptions from the rest of your code. After all, exception handling requires very limited functionality: you need to throw an exception, watch for it, and handle it somehow.

I'm going to describe a system I use. The description is fairly long, but actually the system is very simple and takes a couple of fields, a couple of scripts and a handful of very simple custom functions.

Continue reading "Handling exceptions" »

Merge Expressions

This custom function combines arbitrary text with field data preserving text formatting styles as much as possible. Short, it works like FileMaker native merge feature with some differences. Here's a screenshot from the sample file:

Merge-Expressions.png

  • The function changes everything between between << and >> to the result of evaluating this as a FileMaker expression. One of simplest expression is a field name, but technically such an expression can contains any valid combination of fields, operators and functions. (If this is too much the function can be downgraded to fields-only variant.)

    Dear <<Name>>Dear John Doe
    the late fee is $<<Amount * 0.05>>the late fee is $6.5

  • The function doesn't change invalid or incomplete expressions.

    Dear <<Nmae>>Dear <<Nmae>>
    Dear <<Name, please noteDear <<Name, please note
    Dear Name>>, please noteDear Name>>, please note

  • The function sets the style of the merged data to the style of the placeholder, or, if the latter has mixed styles, to the style of the 1st “<” character.

    Dear <<Name>>Dear John Doe
    Dear <<Name>>Dear John Doe

    If the expression itself has text styles (for example, if it references a field which is manually formatted), these styles are combined with the styles of the placeholder. If the styles conflict and cannot be combined, the expression style is used. For example, if the Name field contains John Doe in blue, then:

    Dear <<Name>>Dear John Doe
    Dear <<Name>>Dear John Doe

  • You cannot set default number, date or time format for the whole piece of text as you can do for FileMaker layout-mode merge feature. If you need to use a special format, you'll have to do this via a custom function or extra field.

Merge Expressions( text )

If( IsEmpty( text ) ; "" ;

  Let( [ 
    S = "<<" ; E = ">>"; 
    start = Position( text; S; 1; 1 );
    end = Position( text; E; start; 1 ) ];

    If( start = 0 or end = 0; text;

      Let( contents = 
        Middle( text; start + Length( S ); end - start - Length( S ) );

        If( IsValidExpression ( contents );
          Left( text; start - 1 )
            & Replace( Middle( text; start; 1 ); 1; 1; Evaluate( contents ) )
            & Merge Expressions( 
              Right( text; Length( text ) - end - Length( E ) + 1 ) );

          Left( text; start + Length( S ) - 1 )
            & Merge Expressions( 
              Right( text; Length( text ) - start - Length( S ) + 1 ) ) ) ) ) ) )

It's a bit difficult to write a test unit test for this function, so I simply made a sample file and tested the function manually. Please tell me if you find an error.

Tips

  • If you want to downgrade the function to use fields only, you can use the GetField() function instead of the Evaluate().

  • If you want to restrict the function to certain fields only, you'd better fine tune this in FileMaker privileges. For example, if you set a field as “no access”, an expression that addresses such a field won't be evaluated.

Technorati tags: , , , , .

Custom functions to simplify reading the current state of modifier keys

The state of modifier keys in FileMaker is represented as a five-bit number taking values from 0 to 31. To read the state of a particular key means to read a particular bit of this number. FileMaker doesn't have native support for binary operations, so developers have to do this on their own. The best way, I think, is to write a few simple custom functions.

The “simplest” (that is without thinking) method would be to go ahead and check the current value of Get( ActiveModifierKeys ) each time you're to read the current state of modifier keys. This approach has several disadvantages. First, one must do it right. Simply checking for

Get( ActiveModifierKeys ) = 4

may or may not give you the state of the Shift key, because at the same moment the user may have the other modifier active. For example, if Caps Lock is active, the result of Get( ActiveModifierKeys ) will be 6; same is for other keys, though they're less likely to be mixed.

Second, it's difficult to check for combinations of keys: the code becomes hardly readable. Time, for example, how long does it take you to answer what exactly is checked by the following code:

Get( ActiveModifierKeys ) = 9

In my applications I use a few custom functions to check the modifier keys' state. For example, I have a function Shift Is Pressed. This function takes no arguments and returns either True or False depending on the current state of the Shift key. It doesn't care whether other modifiers are pressed or not. With such functions the calculations I write look much simpler:

If( Shift Is Pressed, ... If( Option Is Pressed and Shift Is Pressed, ... If( Option Is Pressed and not Control Is Pressed, ...

Continue reading "Custom functions to simplify reading the current state of modifier keys" »

FileMaker field naming conventions

I have been developing FileMaker applications for quite a long time (about 8 years) and have worked with different field naming conventions. Now I stick to the one I consider the best of all: natural field names.

I used to work, for example, with naming conventions that required me to name fields after their types (global, calculation), certain options (lookup) or roles (key). There were also standardized prefixes, so field names were quite cryptic like OopsThingAMaBobKeyGlob.

I hated this system. It wasn't difficult to understand and thinking up a name for a field was also very simple, because there was very little to think up, but in effect all the names looked same to me. I wasn't able to remember any of them and had to always pick them up from lists, which is a pain for a fast typist like me, especially when the typist is trying to write a complex FileMaker-4-5-6-style calculation.

I used to work with my own system too. I thought it would be fine to leverage the alphabet sorting as the only available field organization tool and designed a system with one-character prefixes to indicate field role. There were prefixes for data fields (d), derivative fields (e), interface-only fields (i), keys (k), script variables (s) and so on. A sample field name would be dName, eSum, iSaveButton, sCounter and so on. The field names were shorter, more readable, and easier to type — I almost never had to pick them from list and even if I had to, I was able to find a field in the list in seconds by typing first few characters of its name.

Unfortunately, the whole approach turned out to be wrong. Yes, automatic sorting is evil. When I sort fields manually, I express certain subtle relations between them: I usually group related fields together, and, for example, tend to put derived fields after source fields, and so on. As I said these relations are subtle, hard to formalize and often vary from table to table and from application to application. If I sort fields automatically, I'll lose all these tiny bits of information that help me to better control the development process.

When I understood this, the rest was straightforward. If I sort fields in what I think is their natural order, I should give them natural names as well. Yes! I should name first name as First Name, sum as Sum, options as Options. The only thing to consider when thinking up a name must be whether it clearly describes the thing and is easy to remember.

What are the advantages?

The more I use the system, the more advantages I see.

  • The code becomes far more readable making it easier to communicate with co-developers. A calculation like Subtotal + State Tax + Local Tax + Delivery Fee + Tax on Delivery Fee doesn't require any additional comments, does it?

  • Natural names help me to leverage certain FileMaker features with no additional effort. For example, if a field requires a value FileMaker shows a dialog like:

    Validation-Dialog.png

    If you use natural names, you may not have to write your own custom message, because the automatic one can already be good enough.

  • Finally, sometimes it's good to expose field names to users. (I don't mean unlocking an application to users, though with new security options the idea starts making sense.) In one of projects I develop I had to give users a way to compose text templates with data placeholders and later, when a template is processed, replace these placeholders with actual data. For example, user needed to be able to write something like “this task was completed by <User Name>” and have it changed to “by John Doe” when the task was actually completed.

    Initially I was going to add a table to store possible placeholders and then write a few functions to replace them in a piece of text. I did it actually. The system required two tables with several table occurrences, and a bunch of fields to display the possible choices, and a script to cache values in a global field, and a few custom functions to do the replacement. I think it was good experience, because at the end I realized I don't need all this stuff.

    Most of the placeholders were simply field references and these fields were directly available in the context I needed to calculate them in. Immediately I refactored the whole piece. At the end I had a list of field names, and a very simple selector:

    Field-Selector.png

    The Event::Created On is what it looks like: a field name. When it is inserted in a field, it gets << and >> at its sides and then a custom function goes through the text and evaluates everything between << and >>.

    The whole module became much simpler, much more manageable and even more flexible and the key part to this were natural field names; I couldn't do this with cryptic geeky ones. By the way, can you guess where the description string below the field name comes from?

    You're correct — this is a comment of the field.

Technorati tags: , , .

Number to Decimal

A simple function to convert a number of any base from 2 to 36 into decimal form.

Number to Decimal( number, base )

If( IsEmpty( number ) or base < 2 or base > 36; ""; Let( k = Length( number ); ( Position( "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"; Left( number; 1 ); 1; 1 ) - 1 ) * base^( k - 1 ) + Number to Decimal( Right( number; k - 1 ); base ) ) )

Continue reading "Number to Decimal" »

Number to Base

A simple function to convert a decimal number to any base from 2 to 36.

Number to Base( number, base )

If( base < 2 or base > 36 or IsEmpty( base ); ""; Let( current digit = Middle ( "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"; Mod ( number; base ) + 1; 1 ); Case( number ≥ base; Number to Base( Div ( number; base ); base ) ) & current digit ) )

Continue reading "Number to Base" »

Use modular XSLT to simplify importing XML data into FileMaker

A simple XSLT file encapsulates the target FileMaker grammar into easy-to-use parameterized templates; once written, the file can power any import task with no changes.

To import XML data into FileMaker you have to write:

  • Elements of the source grammar;
  • Elements of the target grammar, FMPXMLRESULT;
  • A list of instructions to transform the former into the latter.

You can put all of this into a single file and it will work; almost all XSLT samples are written this way. But as you face more and more complex import task, you start seeing the drawbacks of this approach:

  • You'll have to repeat all the verbose syntax of the target grammar for every XSLT file you write;
  • A mix of source and target elements and XSLT instructions can become hardly readable, that is error-prone and difficult to manage.

The universal way to manage complexity is to break something complex into modules that hide all the complexity beneath a simple interface. Let see how it can be done with XSLT.

Continue reading "Use modular XSLT to simplify importing XML data into FileMaker" »

Use the Gradient() function to apply a smooth gradient to a string

The Gradient( text, colors ) custom function paints the specified text with colors, applying a smooth gradient if more than one color is specified.

Sample.png

It may be not be terrible useful but looks nice and is fun to study as an example of a recursive function.

Technorati tags: , , , , .

Continue reading "Use the Gradient() function to apply a smooth gradient to a string" »

Assert Equals(): a custom function to test other custom functions

This function is designed to test other custom functions using the extreme programming approach and, most probably, a custom function test unit (if you haven't read the referenced post, you'd better start with it). The function works fairly well both as a specification and as a test engine.

Technorati tags: , , , , .

Continue reading "Assert Equals(): a custom function to test other custom functions " »