The Filemaker platform has a huge number of benefits to both end users, and to developers. Over the 25 year history of the platform, many features have been added along the way that have drastically improved our options and possibilities for delivering functionality.
BUT.
We should all by now realise that just because you can, it doesn’t automatically follow that you should. Todays topic will evaluate one part of the Filemaker developers arsenal, first added in Filemaker v7 - the Container field type. This new field type allowed the storing of any file data in a field, which could then be used in layouts, scripts, and calculations without any need for extensions plugins. This allowed for the embedding of PDF’s and Images to be stored alongside traditional data, and users loved it.
Fast forward a few years (decades?) and the humble container field remains, but is it still relevant? One of the main concerns with container Data is the amount of space it takes up. All things being equal, even a database with millions of records and dozens of tables can still be only a few hundred MB in size without container Data, and this means that the storage space for both the live files and the backups copies of your files can be sourced relatively cheaply. However, once you start adding container Data, this can easily increase to a point where the amount of space each file takes up becomes unwieldy. Not only are storage costs increased, but the time each backup takes can start to become an issue, even if you use remote container storage.
Fortunately, we can now look to store container Data outside of our Filemaker files, but still have it instantly available within our solutions, and we can build this in such a a way that for that vast amount of use cases, the end user is simply not aware of where that container Data is stored, it just works.
Enter: Amazon S3 (Amazon Simple Storage Service) - used by huge swathes of the web to store and deliver content.
A lot of the recent work we’ve been doing in this area leverages the excellent blog post and accompanying custom function by Jason Wood here:
https://www.definedatabase.com/s3-object-storage-filemaker-container-alternative/
I won’t dwell too much on the technical elements of this implementation, suffice to say that using a custom function for creating the pre signed url format required for s3 calls, and the web viewer to render the files, we can move this container data off to S3 compatible storage.
So - WHY would we want to do this, and what are the caveats?
Requirements / caveats:
Really we’re going to want to be running Filemaker v19, and it should go without saying that client devices will need an internet connection.
Also - this kind of approach tends to work best if you have already separated your container data across your solution into a single table, which is then throughout your solution via portals etc.
The two main driving forces behind this are cost reduction, due to the reduced need for disk space, and the performance of backups on the server. If neither of these things are of concern to you, this might not be an exercise worth undertaking.
I won’t go through all of the steps required to do this here, as they’re really well outlined in Jason’s post listed above, but there’s a couple of notes I’ll add which I found useful:
- Keep all Container Records in a single table, one container field per record. This makes things a lot easier to manage.
- Build a standardised card window layout to display container data. You write a single script to load this window and display data on whatever parameters you pass into it, and it will mean you only have one place to change things.
- Consider creating a thumbnail of container data that you’re uploading to S3, as this might be useful for displaying a preview in portals, which do not support displaying webviewers.
- Take the time to understand the implications of outsourcing your storage to S3 - there are a huge range of options you can implement around retention / replication / access, so do yourself a favour and really get to know and understand what your options are.
- Remember - the S3 data will be completely independent of your Filemaker file, so if you delete a container from a development copy of your file, and you’re not careful about it, you can also delete the object from s3, which will render it inaccessible from your live file!
