Allgood.systems on David Hamann

allgood.systems: get a Slack message when your server goes down

Sun, 28 Jul 2024 00:00:00 +0000

I was recently asked if it’s possible to post a message to a Slack channel whenever the state of a monitor on allgood.systems changes (for example, a web server goes down, a job starts failing, etc.).

The answer is “yes”, and it’s quite easy to set this up once you have a Slack app configured.

All you need to do on allgood’s side is to add a web hook to your notification group, enter the URL you get from Slack, and define what message you want to send.

Creating a Slack app

In the past you could just add Slack’s “Incoming WebHook” to your workspace and were – more or less – done.

A while ago, however, this method was deprecated, and Slack now recommends to create your own “App” instead.

“Creating an app” sound more involved than it actually is. You just have to click through a few menus to “build” one.

Let’s follow the Quick Start Guide and take the following actions:

Go to Apps and click “Create New App” -> “From scratch”. Then give your app a name, for example “allgood bot”, and choose the workspace to use it in.

Go to “OAuth & Permissions” -> “Scopes”, and then add the following “Bot Token Scopes”: chat:write.public and chat:write. This will allow your bot to post messages.

Go to “Incoming WebHooks” and set it to “enabled”.

Go back to “Basic Information” and click “Install to Workspace”. You will be prompted to allow the bot’s actions and have to select a channel for your bot to post in (for example #general).

Go back to “Incoming WebHooks”, and copy the URL that is referenced for the channel you selected.

Now, log back into allgood.systems and continue the setup there.

Setup on allgood.systems

First, you go to notification settings and then either select an existing notification group or create a new one.

For the existing or new group, you can then click on “Add member”, select “Web hook” as type and “POST” as method:

Now, you can copy the web hook URL from Slack and enter it into the URL field.

In the JSON payload you can define your desired message structure and make use of the available placeholders (such as the monitor url).

For example:

{
 "blocks": [
 {
 "type": "section",
 "text": {
 "type": "mrkdwn",
 "text": "{{subject}}"
 }
 },
 {
 "type": "section",
 "text": {
 "type": "mrkdwn",
 "text": "👉 <{{monitor_url}}|View Monitor>"
 }
 }
 ]
}

If you’ve attached the web hook to a notification group already linked to your monitors, your setup is now complete.

If you instead created a new notification group, go to one of your monitors, select “Edit” and choose the new group in the monitor’s notification settings.

Now, whenever a notification is triggered, you will see a new message in your configured Slack channel.

And, if you enabled repeated down notifications, you will also get these notifications repeatedly in Slack.

allgood.systems: Monitoring the duration of your background jobs

Tue, 26 Sep 2023 00:00:00 +0000

Since the launch of allgood.systems you were able to monitor if your background jobs, scheduled tasks, cron jobs, etc. were running whenever you expected them to be running.

This was accomplished by defining an interval on allgood and then sending a request to a check-in URL at the end of your script.

If the request didn’t arrive at the end of the specified interval, you would get a notification that your job is down.

This still works the same way. But from now on you can also choose to send an additional request at the beginning of your jobs.

This allows you to get an idea how long your jobs typically run (average, minimum and max are being displayed).

You can learn more by reading the documentation or just trying it out.

The feature is live as of now.

As always, let me know if you have additional ideas that could be helpful.

Monitoring FileMaker scheduled scripts

Fri, 12 Aug 2022 00:00:00 +0000

In this tutorial I want to describe how you can setup immediate notifications whenever your scheduled FileMaker scripts stop running – for example due to a crashed FileMaker scripting engine, an error in your script or just general server downtime.

I will be using allgood.systems, a monitoring platform I recently built.

Creating a new Job Monitor

Once you have registered on allgood.systems you can navigate to the “Job Monitors” tab to create a new monitor for the FileMaker script you would like to get notifications for.

We can give the monitor a name for reference and then set a check-in interval.

Let’s say our script is scheduled to run every 60 minutes. Accounting for some delays and runtime, it makes sense to define a custom check-in interval of 65 minutes.

Our plan here is to get notified if and when the scheduled FileMaker script does not run in a timeframe of 65 minutes.

Leaving the other options as default, we can click “Save”.

Integrating the check-in URL

After adding the monitor it will have a “Started” status. This means it is active but has not yet received a check-in or detected any downtime.

Clicking on the monitor will reveal its “check-in URL”. Copy the link to your clipboard and then head over to your FileMaker script.

Generally, it makes sense to let your script check in after it has successfully performed its tasks. Let’s go ahead and add an Insert From URL step near the end of the script.

For our demo monitor configured above this step could look like this:

# check-in at allgood

Insert from URL [ Select ; With dialog: Off ; Target: $result ; "https://ping.allgood.systems/l/j/phY3K0ADSMadoohzFs0Xaw" ; Verify SSL Certificates ] 
Exit Script [ Text Result: True ]

Every time your script is run you should now see a check-in entry on allgood. And if there’s ever the situation where no check-in happens in the defined interval (no matter what the cause is), you will get a notification based on your settings on allgood (an email by default).

Want more?

In the same way as described above you could also add so-called “Trap Monitors”. In this case you would place the URL-triggers in places which you think are executed very rarely (for example edge-cases in your scripts). If those rare cases happen, you want to know immediately. Thus, with “Trap Monitors” you will get a notification on every (!) check-in.

Additionally, to just monitor the uptime of your FileMaker Server itself, you can do so with “Net Monitors” – either by checking if the server responds on port 5003, if the WebDirect/Data API/welcome page responds as expected or if your domain resolves successfully. As an extra you will also get reminders when your SSL/TLS certificates are about to expire. See the documentation for more information.

Have any more questions? Feel free to reach out.

Monitor websites and detect when cron jobs and scheduled tasks are not running

Sat, 09 Apr 2022 00:00:00 +0000

TL;DR Want to monitor your websites or get notified when your cron jobs or scheduled tasks are not running when they are supposed to run? Check out https://allgood.systems.

For quite some time I was planning to build some piece of software that notifies me when a web app goes down, returns unexpected results, or changes content. In addition – since it’s often hard to keep track of all the background jobs that are running – I needed and wanted something that informs me when my services stop (!) doing what they are supposed to do in regular intervals (which is often detected much later and in turns makes recovery/clean-up much harder).

While I was thinking through the problem I also noticed how frequently in the past I received reports about applications that apparently stopped working from one day to the other. Often the reason turned out to be that someone forgot to renew a TLS certificate and no automated renewal was set up (while automation is generally better, often it’s just not done or you are not in control). So getting notifications about upcoming SSL certificate expirations was another thing that fit into the scope of the project.

Finally, I wanted a service which could quickly give me a private HTTP endpoint and alert me instantly whenever it received any request – for example to get informed about rare events, check when some page or file is opened, test for vulnerabilities, etc.

With this concrete need and a desire to learn a couple of new technologies along the way, I built a service that now addresses the above use cases. If you like to try out an early version of it, feel free to head over to https://allgood.systems and sign-up. With some limitations you can use it free of charge. For usage examples, please also see the documentation.