Gerrit Niezen

Maker of open-source software and hardware.

Feature image

OK, this is the last of my “favourite X of the first half of 2018” posts, this time covering some TV series I've enjoyed watching. Lately I mostly just watch Netflix, and prefer reading to watching TV, so I definitely don't consider myself to know what's good on TV.

  • Altered Carbon - I read the book a while ago, and obviously the TV series doesn't match up to the book. Still, it's entertaining viewing, if you don't mind your sci-fi being a bit violent.
  • Black Mirror Season 4 - The best kind of sci-fi is the kind that you think about long after you've first watched it.
  • Star Trek: Discovery - It may be a different than the other Star Trek series, but still very enjoyable.
  • Safe - Hearing Michael C. Hall speaking with a British accent takes some getting used to, but it's definitely worth the watch (even if you're not a fan of Dexter).

Currently I'm watching The Staircase, but I'm not yet sure whether to recommend it as I'm just over halfway in the season.

Comment on this post

Feature image

Wow, I didn't think I'd be able to keep this blog going for two months without skipping a day, but here we are. As I'm not using any analytics on this blog, I have no idea how many people are actually reading this. Which is probably a good thing, as my focus is on just putting something out there every day and not necessarily trying to build an audience.

I hope someone finds some of the things I write here useful. Maybe I should at least enable a comment section to give anyone reading this the opportunity to provide feedback. You know what? I'm going to add it right now.

Comment on this post

Feature image

Today I started on wrapping the libmtp C library in a Node.js module. So far, I've managed to load the list of files on the device and successfully retrieve a file.

Let's start with the bindings.gyp file:

{
  "targets": [{
      "target_name": "module",
      "sources": [ "./src/module.c" ],
      "libraries": [
          "<!@(pkg-config --libs libmtp)"
      ],
      "cflags": [
          "<!@(pkg-config --cflags libmtp)"
      ]
  }],
}

Here I'm specifying that the file containing the C code is in module.c, and that we're using the libmtp shared library. On macOS, use brew install libmtp. On Linux, use sudo apt-get install libmtp-dev. Oh, you'll also need pkg-config if it's not installed on your system yet[1].

Here is the NAPI function to run the code in the C library and return the result to JS:

napi_value getFile(napi_env env, napi_callback_info info) {
  napi_status status;

  size_t argc = 2;
  napi_value argv[2];
  status = napi_get_cb_info(env, info, &argc;, argv, NULL, NULL);
  if (status != napi_ok) {
      napi_throw_error(env, NULL, "Failed to parse arguments");
  }

  int id = 0;
  char path[20];
  napi_get_value_int32(env, argv[0], &id;);
  napi_get_value_string_utf8(env, argv[1], path, 20, NULL);

  int ret = LIBMTP_Get_File_To_File(device, id, path, NULL, NULL);

  napi_value retVal;
  status = napi_create_int32(env, ret, &retVal;);
  if (status != napi_ok) {
      napi_throw_error(env, NULL, "Unable to create return value");
  }

  return retVal;
}

For details on how we get the arguments converted from JS to C, and the return value back to JS, see this tutorial. The line to focus on here is the one with LIBMTP_Get_File_To_File, where we pass the file ID to get from the device, and a path to save that file to.

As we need to connect to the device before we read the file, I added the following code to the NAPI Init function:

napi_value Init(napi_env env, napi_value exports) {
  napi_status status;
  napi_value fn;

  LIBMTP_Init();

  fprintf(stdout, "libmtp version: " LIBMTP_VERSION_STRING "\n\n");

  device = LIBMTP_Get_First_Device();
  if (device == NULL) {
    printf("No devices.\n");
    return 0;
  }

  status = napi_create_function(env, NULL, 0, getFile, NULL, &fn;);
  if (status != napi_ok) {
      napi_throw_error(env, NULL, "Unable to wrap native function");
  }

  status = napi_set_named_property(env, exports, "get_file_to_file", fn);
  if (status != napi_ok) {
      napi_throw_error(env, NULL, "Unable to populate exports");
  }

  return exports;
}

NAPI_MODULE(NODE_GYP_MODULE_NAME, Init)

The line with napi_set_named_property specifies what the function name is called in JS, while napi_create_function creates a JS function from the C function called getFile.

Now it's as easy as just calling the function from JS:

const binding = require('node-gyp-build')(__dirname);
    
console.log(`Status:`, binding.get_file_to_file(1693, 'test.jpg'));

We pass the file ID (1693 in this example) and the path to save file into (test.jpg in the same folder). And that's it, we can read files from an Android phone over MTP on macOS! I'll be posting all the code on GitHub soon once I've cleaned it up a bit.


  1. brew install pkg-config on macOS. ↩︎

#nodemtp

Comment on this post

Feature image

Following up from yesterday, I started using libmtp to see if I can connect to an Android phone and read a file from it. Once the library is successfully compiled, you can run some of the example programs in the examples/ folder. To list all the files on your device, run ./mtp-files. Then, to retrieve a file, you can run

./mtp-connect --getfile <FILE_NUMBER> <DESTINATION>

It seems to work just fine, so the next step is to wrap it as a Node.js library. There's a great article by Mathias Buus on the past and present of handling native (C/C++) modules in Node.js. Basically, N-API is the way to go. It's already stable in Node v10, and works in Node v8 as well.

I followed this tutorial to build a basic native module with N-API, and it works great. The best part was building it and running it in Node v10, and then running the same code in Node v8 without any issues. Nice!

#nodemtp

Comment on this post

Feature image

Media Transfer Protocol (MTP) has replaced USB Mass Storage as the dominant protocol for transferring digital media between an Android phone and a computer. USB Mass Storage allows devices to be mounted as drives on the host computer, and while MTP also allows drives to be mounted, it happens at a level where it's easier to restrict which files are made available to the host system. If that sounds to you like something Microsoft would design, you'd be right.

MTP works great on Windows machines of course, and surprisingly well on Linux. However, while there's apparently some basic MTP support on macOS, it doesn't even allow you to mount a device without external software, like Android File Transfer.

I spent some time today looking at options for connecting to an Android phone on macOS with MTP using Node.js. Here are some possible options:

  1. Use FUSE (Filesystem in Userspace) with Node bindings. This will requires installation of FUSE for macOS first, which I think also involves installing a kernel extension. So it's basically a no-go for macOS High Sierra and above.
  2. Write a wrapper for libmtp, using this unmaintained repo as a base.
  3. Port parts of this application over to Node.js, as its only dependency seems to be libusb. This means essentially writing your own MTP implementation on top of node-usb.

Since MTP has been standardised as part of the USB Device Class specifications, writing an MTP implementation on node-usb does make sense, but will probably be a lot of work. I think I will opt for (2) first as it's easier, with (3) as a fallback.

#nodemtp

Comment on this post

Feature image

If you have an e-reader/Kindle, and you haven't yet had a look at Standard Ebooks, do yourself a favour and go there now.

Depending on where you are in the world, copyright only expires 50 to 100 years after the creator's death – crazy if you really think about it. In any case, that means hat you won't find any recently written books on Standard Ebooks, but you will find some great classics. Here are some that I still want to read, in no alphabetical order by author:

#Reading

Comment on this post

Feature image

This afternoon we took my son to a soft play area close to our house. He's around 18 months now, and going through a bit of a sleep regression, so we though it would be good to tire him out so that he gets a good night's rest[1].

He quickly got bored of the toddler play area, which consisted of a ball pit, a couple of crawling areas and a small slide. I decided to take him to where the big kids play, where he can go as long as he is accompanied by an adult the whole time. This specific play area is known for their big slides.

There are some really scary slides, even for an adult, where there is an almost vertical drop for a couple of meters before it evens out. Obviously I couldn't take him down those ones, but there were a few open and closed slides where he could sit on my lap and we go down together. And boy, did we enjoy them!

He ended up running around the big play area with me following behind, climbing up and then going down the slides some more. We even found a pair of small slides where could go down at the same time, which was good for a laugh. When you're an adult, you sometimes forget how much fun a simple slide can be. Luckily kids are there to remind us.


  1. and us too of course! ↩︎

Comment on this post

I'm a little bit behind on my 2018 Goodreads Reading Challenge, but as we're now a month past the halfway mark of 2018, I thought I'd do a quick roundup of my favourite three books of the first half of the year:

  • The Divide, by Jason Hickel: I think about this book very often. It clearly explains why there is global inequality and that development aid is not the solution.
  • The Year of Less, by Cait Flanders: A delightful little book that describes our struggles with consumerism, why we do it and how to stop.
  • Jupiter War, by Neal Asher: the final book in the Owner trilogy, a heavy sci-fi series full of chaos and destruction. Not for the faint of heart.

“In the US, the portion going to the top 1 per cent more than doubled from 8 per cent in 1980 to 18 per cent today. Britain witnessed a similar jump during this period, with the share claimed by the richest growing from 6.5 per cent to 13 per cent.” ― Jason Hickel, The Divide ―

#Reading

Comment on this post

It's been more than two weeks after my last update on getting the USB host shield working with Espruino. I decided to work on it a bit more yesterday so that it doesn't end up in the abandoned projects drawer.

After I discovered earlier this week that the TUSB3410 chip won't work unless you upload firmware into it first, I decided to change tactics and attempt connecting to a different chip instead. A week ago I got my Node.js user-space driver for the Silicon Labs CP2102 chip working[1]. This chip is inside the OneTouch VerioIQ glucose meter, so I plugged the meter into the USB host shield and started swapping out the TUSB3410 code for CP2102 code.

It again didn't work at first. I connected the logic analyser and noticed that there still wasn't any USB bulk in transfers happening, even though everything sent was being acknowledged by the device. However, there was a glimmer of hope as I discovered some data packets in the bitstream that looked like what I was expecting to receive. So the data was coming back, but my bulk in transfer wasn't created correctly to capture it.

After looking through the code some more, I came to the conclusion that the issue was with the data toggles used by the MAX3421. I finally figured out that I need to keep track of separate toggles for each endpoint, and as soon as that was in place I was greeted with this:

verioSuccess

I was able to see data being returned in a bulk in transfer! Amazingly, I'm finally getting data back from a device connected to the USB host shield on my Espruino Pixl.js. I guess this could've happened much sooner if I went with the CP2102 first instead of the TUSB3410.

Now, the next step is to get some actual blood glucose data from the meter and visualise it on the Pixl.js screen. If all goes well, that should be much easier!


  1. now available on GitHub and as an npm package ↩︎

Comment on this post

Feature image

After I wrote yesterday's post, I decided to give it one last shot and at least try and implement the workaround. And guess what, it worked! I can successfully connect to a TUSB3410 chip on macOS using Node.js, update its firmware and read data from the device it's connected to. Woot!

I have to say, being able to finally use Wireshark on macOS with High Sierra made things a lot easier. Without USB traces from Wireshark I wouldn't have guessed that the device wasn't being reset by macOS after the firmware was updated.

It seems that libusb_reset_device has been broken on macOS since El Capitan. Changing

kresult = (*(dpriv->device))->ResetDevice (dpriv->device);

in the libusb_reset_device function to

kresult = (*(dpriv->device))->USBDeviceReEnumerate (dpriv->device, 0);

works great. It requires recompiling libusb in node-usb and then running npm install --build-from-source in node-usb. I've reported the bug to libusb, so let's wait and see what they say.

Comment on this post

Enter your email to subscribe to updates.