where is the bus?

i have a car now! 🎉

i live somewhere in which having a car is basically required (boo), but it took until i was 23 to finally get my license sorted, and then till 24 to actually get a car

for the longest while i managed to get by with:

• my own two feet and a good pair of shoes
• bumming rides off of friends ❤️
• maybe a taxi or two here and there

however, while there is a bus system here…

…its,

meh

given that st. john’s is quite old, its not surprising its streets were designed way before the modern motor vehicle, hence a lot of the bus routes here runs are… interesting

but, the routes exist, you can make your way from point a -> b in the city and (some of) the surrounding areas

and thankfully the modern ability to use technology to navigate the bus service does indeed exist!

unfortunately, while you can use apps like google maps to view routes and time tables for busses, you cannot see realtime bus information within said map apps

i’ve been to bigger cities with this sort of integration, or used transit that had it, but alas, that is not available here…

…or is it

it is!, sort of

the company behind bussing here, is Metrobus, and they have their own app for getting all sorts of information about bus statuses and such on the go

similar to the service they provide however

its meh

here you can see the google play store page for ‘m-Mobile v2’, alongside its home screen

behold, truly the pinnacle of modern app design

the application has a few features, most of which are useless if you’re using a map app already (i think some of the features here just use google maps under the hood)

what it does have, that map apps lacks, is realtime bus location data

heading into the timetrack page, you can see a list of the busses servicing different routes

the first screen tells you, for each bus:

• is it late?
• how late is it?
• where is it currently?
• what route is it servicing?

you must then click into the list item for the specific bus you care about, to get a page that contains:

• the same information as before
• but with a map showing current bus location (with no visible information of when this location was updated, it could have been 5 seconds ago, or 5 minutes ago)
• and the route that this bus is taking (without the current location of yourself visible on said map)

this view… gets the job done

but, from a user experience perspective, i’ve always felt like there is a better way to display this information in a much more concise interface

i’m also a map guy: this data should all be on a map

but hold up, how does one go about fetching this data?

this is a view in a mobile application, do i have to rip out adb and a decompiler to find strings for the api that backs this page?

nope!

i forget when i saw a link to metrobusmobile.com, i think it was on the metrobus website itself, but quickly, you’ll find out that the app, is just a webview

so… this could have just been a PWA, but of course its instead an entire app you’re asked to download from the app store instead

but, as far as data digging goes, this means one can pop open this same tab within a desktop browser…

…and dig their dirty fingers directly into the dev tools, to hopefully see what endpoints are being reached out to populate these pages

now i’ve done some digging, and it seems this is an ASP.NET application, with some routes being rendered on the server, but other routes sprinkling in some javascript + apis, usually for pages that they want realtime functionality on

thankfully, this includes the timetrack page!

all one does to get a blob of json used to populate this screen, is hit this endpoint:

curl https://www.metrobus.co.ca/api/timetrack/json/

a simplified version of the json this spits out follows:

[
  {
    "routerun": "Rt 1-1",
    "current_route": "01-1",
    "routenumber": 1,
    "vehicle": "1524",
    "time_stamp": "2:12 PM",
    "deviation": "1 MIN BEHIND",
    "service": "1",
    "current_location": "Cashin Avenue civic  122",
    "bus_lat": "47.55193",
    "bus_lon": "-52.72940",
    "heading": "NNW",
    "speed": 24,
    "position_time": "2:12 PM",
    "position_time_seconds": 51150,
    "departure_time_at_closest_stop": "51060"
    ...
  },
  ...
]

(i removed some fields that were the same data / all the same basically null or boring value)

so cool, this is the data that’s used on both the timetrack list page, and the inner pages!

what’s nice is that we actually do get the geospatial information that the inner page uses for every bus, so we only need to make one request to get all this data

now that i know i can get the data… what’s next?

6 years ago, i asked this same question, and built metrobus-spy

metrobus-spy was basically just a proxy that would hit the timetrack json endpoint, clean up the data a bit to just contain latitude, longitude, and the position_time values

this api, didn’t really do much at all, my friend riley threw together a little project that fed from metrobus-spy and generated some heatmaps, as you can see here:

skip ahead to september of 2022, i start a new repository, inspired by a concept i learned from Simon Willison’s Weblog, to hit the timetrack api every 5 minutes, and store the result into said repository

the setup explained by simon is interesting, it uses a very simple cron-scheduled github action that performs whatever fetch you want, and then commits to the repo if the data returned from that endpoint has changed

this means since this is a public repo, and actions are free on public repos, i don’t pay any fees! (the only issue is if the repo grows in size that it becomes a problem, but i’m just not going to think about that for now…)

i have figured out ways to turn this data into a sqlite database, and then a Parquet file via DuckDB, but done no future science with the data so far

if this tickles your fancy / you want to use this data for your own project, go right ahead!

this year, i did end up taking a stab at building my ‘envisioned’ version of the user interface, very much not exactly to the point i’d expect it to for others to actually use it, but very much a proof of concept of how the data can be layed out:

during a weekend, some people i teamed up with during a local hackathon (organized by mitch ❤️) built together a remix application that used mapbox gl js to plot all the busses

clicking on the busses showed their route, and the map centers itself on where you are not where the bus is, which even though this application is super bare bones, it honestly feels in many ways an instant improvement over the timetrack page

more work would have to go into this to make it better, maybe someday i’ll get back to it, but if this project seems like something you’d like to dig into: prs welcome

all in all, i figured out some time ago how to answer the question “where are the busses?” with code, and threw a few projects together

but yeah, i have a car now lol

any thoughts about any of the above?

reach out: