Uploading files to FileMaker Server without a Pro client

Back in the dark ages the FileMaker Server admin console (then Java Web Start) allowed you to remotely upload new fmp12 files to the server. For some reason this feature did not survive the admin console rewrite a decade (?) ago. Rather, the upload feature was integrated into the Pro client. Later, as the Admin REST API was released, the upload feature was still missing.

The only two options to upload fmp12 files to the server are thus:

• use the Pro client to upload
• upload by copying files directly onto the server (manually or scripted)

A couple of days ago I wondered if there’s any technical reason the Admin HTTP API doesn’t support this feature or if the Pro client does something special. So I decided to have a quick look at it – and it turns out, FileMaker Pro is actually just using an internal HTTP API.

Looking at the wire

I started the FileMaker Pro client, selected the Upload feature and listened to the network traffic.

By default, all you see is TLS traffic which isn’t so helpful. Since the traffic was going to port 443 rather than FileMaker Server’s default 5003 I assumed this was just encrypted HTTP traffic.

I had my test FileMaker Server installed on Linux, so I checked the SSL settings in the nginx config at /opt/FileMaker/FileMaker Server/NginxServer/conf/fms_nginx.conf as nginx receives all the traffic on port 443.

nginx actually gives the traffic to the fmserverd process which has a listener on the loopback interface on port 1895.

In the config I relaxed the security settings a bit so that I can easily decrypt the traffic on the fly. This meant:

• Disable the TLSv1.2 and TLSv1.3 protocol
• Change the cipher to AES128-SHA (to not have to worry about forward secrecy)

# ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers AES128-SHA;

Then I imported my private key that I generated for my test server into Wireshark (Preferences -> RSA Keys), restarted the web server (fmsadmin restart httpserver) and listened again to the traffic.

There are other ways of doing this, but at the time this seemed like a quick way to get to the result.

Analysing the traffic flow

Being able to now follow the HTTP stream in Wireshark made it quite obvious how the communication for the upload worked.

When opening the Upload dialog in FileMaker Pro, the client asks the server for some basic info:

GET /fmws/serverInfo HTTP/1.1
Host: the-server
Accept: */*

The server then replies with a json structure describing the installation (version, settings, license type, etc.). It also sends a public RSA key (more on that later).

{
	"data" :
	{
		"APIVersion" : 1.0,
		"AcceptEARPassword" : true,
		"AcceptEncrypted" : true,
		"AcceptUnencrypted" : true,
		"AdminLocalAuth" : "on",
		"AllowChangeUploadDBFolder" : true,
		"AutoOpenForUpload" : false,
		"DenyGuestAndAutoLogin" : "false",
		"Hostname" : "the-server",
		"IsAppleInternal" : false,
		"IsETS" : false,
		"PremisesType" : "0",
		"ProductVersion" : "13",
		"PublicKey" : "-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQ<snip>\n-----END RSA PUBLIC KEY-----\n",
		"RequiresDBPasswords" : true,
		"ServerID" : "<id>",
		"ServerVersion" : "19.6.3 302(12-29-2022)"
	},
	"result" : 0
}

Once you add your server credentials in Pro (still in the Upload dialog), the client authenticates to the server using your provided username and password in encrypted form (note that this traffic is normally already encrypted and not readable by a third person).

GET /fmws HTTP/1.1
Host: the-server
Accept: */*
X-FMS-Command: authentication
X-FMS-Encrypted-Username: <base64 encoded ciphertext>
X-FMS-Encrypted-Password: <base64 encoded ciphertext>
[...]

The server replies with a session token which is subsequently added as a header (X-FMS-Session-Key) to all requests:

{
	"data" :
	{
		"sessionKey" : "C1DC02CB2D441388DC7956D9B3863D61"
	},
	"result" : 0
}

The client goes on to get the database folder list, files and available free space (all via /fmws endpoints).

Once you select your database file in the client and eventually click on “Upload”, a PUT request is issued to the server (here for the main DB folder):

PUT /fmws/MainDB/UploadTemp_FMS/C1DC02CB2D441388DC7956D9B3863D61/test.fmp12 HTTP/1.1
Host: the-server
Accept: */*
Content-Type: application/octet-stream
X-FMS-Command: upload
X-FMS-Append-Checksum: false
X-FMS-Session-Key: C1DC02CB2D441388DC7956D9B3863D61
Content-Length: 282624
Expect: 100-continue

<the fmp12 file's contents>

There are a couple of more requests to send a start and end event, a request to open the database, a status check and so on, but essentially this is all there is to uploading a database. If your app uses remote containers, these are sent as well to the /fmws/MainDB/UploadTemp_FMS/ endpoint (followed by the container folder structure).

Building my own client

While the above may sound quite involved, it takes a relatively short time to figure out the relevant requests and responses when you control all the components (the client, the network and the server). So I decided to try to build my own “quick and dirty” client to be able to upload files.

Once I had the basic structure of the requests, I could easily experiment with different headers and payloads. This was helpful to figure out one important thing: how does the client encrypt the username and password (the credentials that are also used for the admin console/api).

Since the server needs to be able to decrypt the encrypted credentials, the client needs to use information the server actually has as well. The ciphertexts were changing with every authentication but were nicely aligned in size. So I just tried to encrypt my admin credentials using the public key which the server gives the client as part of the step before the authentication (via the /fmws/serverInfo endpoint). This worked and I got a valid session token back.

The public key is stored on the server at /opt/FileMaker/FileMaker Server/CStore/machinePublicKey and obviously differs from installation to installation.

With this information I had everything I needed to replicate the HTTP traffic for uploading a file.

On GitHub you can find the script that let’s you upload a single file. If all works, the output looks like this. If not, the script likely crashes.

$ python3 upload.py
Usage: python3 upload.py <host> <username> <password> <file>

$ python3 upload.py host.example.com admin password test.fmp12
INFO:Getting RSA public key
INFO:Getting session token
INFO:Got session token: 7EF28EA9E1C9AADDEF745E0B576C84B4
INFO:Sending upload start event
INFO:Uploading file
INFO:Upload done
INFO:Sending upload end event
INFO:Sending open database command
INFO:Checking status
INFO:DB open
INFO:Done

Word of caution: As should be clear from the post, my custom client is the result of me figuring out an undocumented API for a couple of hours in the evening. This is obviously not a stable solution to handle your precious database applications :-) It also only uploads to the MainDB folder, skips the checks the FileMaker Pro client does, and has no support for remote containers (but could all be added). If you want a stable solution, please use the official Pro client or copy files directly onto the server.

Conclusion

I don’t really have a conclusion to this for now, but it’s nice (and potentially useful) to know that there’s a HTTP API on the server for direct database uploads that does not require you to have access to the whole server system but also does not really require a full-blown Pro client.

Why is it not part of the admin API or console? It could be a variety of business or technical reasons and speculating about it is probably not a very useful way to spend time :-)