When email in a .msg file is not ASCII
Sep 18th, 2017 by miki

Got myself an exported message from (apparently) Exchange/Outlook in a file with .msg extension. My initial thought was that this was just a plain ASCII email (seem to remember having handled .msg files as such at some point), but looking at it as text exposed a load of binary and the “file” utility reported it being of type “Composite Document File V2 Document, Cannot read section info”.

What is it?

Some searching reveals .msg is in a proprietary format called “Outlook Item (.msg) File Format” (or formally, MS-OXMSG, find the specification  in the MSDN document entitled “[MS-OXMSG]: Outlook Item (.msg) File Format“). This format is related to the CFBF ([MS-CFB]: Compound File Binary File Format).

Check out what Library of Congress has to say about .msg files.

Interpreting it


The GNOME project has developed an open source library and some tools for interacting with files of this type, check out libgsf documentation at and source code at Or install the package named libgsf-bin in Debian/Ubuntu for access to the “gsf” binary (man page) that can inspect and dump contents from a .msg file. This is quite rudimentary if you just want to read the content.

MSGViewer (java)

MSGViewer is a stand alone java application that can display/convert different mail-format, including .msg. On GitHub I found a fork which claims to provide further features than the original.

msgconv (perl)

msgconv (github source) is a perl script that converts a .msg file into standard RFC822/2822/5322 format .eml (email) format. Install as package “libemail-outlook-message-perl” on Debian/Ubuntu.

$ file test.msg
test.msg: Composite Document File V2 Document, Cannot read section info
$ msgconvert test.msg
$ file test.eml
test.eml: UTF-8 Unicode text, with very long lines, with CRLF line terminators

I prefer the approach of using a standard format for storing data, so I ended up accepting the .eml from msgconv which I propagated to other recipients of the file.

Generating passwords for Mosquitto MQTT broker using PHP
Jan 13th, 2017 by miki

Here is a delayed writeup of a my involvement in a question posted to the liberally licensed MQTT broker (server) Mosquitto’s developer list list about how to generate authentication tokens programmatically. It kicked the curious cat in me which propelled a journey into the backyards of the C code for the mosquitto_password tool which normally is used for this purpose. This resulted in the proof of concept PHP implementation outlined in my answer on the list which is reproduced below.

MQTT (once Message Queue Telemetry Transport) is a lightweight publish/subscribe protocol intended for communication between low power, low bandwidth embedded devices. These days it is commonly hyped as a holy grail in the religion of IoT. The protocol was originally developed by IBM but is now a standard overseen by the OASIS standardization organization which also has the OpenDocument standard (ODF, think Open/Libre-Office) under its wings. According to Wikipedia MQTT is used behind the scenes of Facebook Messenger, OpenStack and Amazon’s IoT services.

For further practical use of the concept outlined you would need to produce a random 16 byte base64 encoded salt to feed into the hasher, that could be done using something like; $salt_base64=base64_encode(openssl_random_pseudo_bytes(12));

If your need a shrink wrapped solution to this you could try to ping me.

Hi Srinivas.

On 2016-07-26 12:49, Srinivas Pokala wrote:
> Username successfully created using linux command with: 
> "mosquitto_passwd /etc/mosquitto/passwd guest".
> I need to create same with php or javascript how?

Looking at the source of mosquitto_passwd
basically all it does to generate the resulting line you see in the
password file is:

1) draw a random 12 byte binary salt
2) hash the combination of password and salt using sha512
3) write username, base64 encoded salt, base64 encoded hash in one line

A PHP implementation would use something like this ($salt is fixed for
demonstration purposes, it ought to be random in production);

$hash=hash("sha512",$password.$salt, true);

Comparing against mosquitto_passwd using a one-liner (uses the base64
salt from output to be able to correlate the two);

$ mosquitto_passwd -b ~/mosq_passwd_test Bitten Insect
$ cat ~/mosq_passwd_test

$ php -r '$username="Bitten"; $password="Insect";
$salt_base64="mfJ0Eq3rIDLKG33r"; $salt=base64_decode($salt_base64);
$hash=hash("sha512",$password.$salt, true);

As can be seen, the PHP generated password line are identical to the
mosquitto_passwd generated.

I have also successfully tested authentication against the mosquitto
broker with PHP generated users. One caveat is that the above can
generate a salt of arbitrary length, but the broker must see a 12 byte
binary salt (16 byte base64) or authentication will fail.

Note however, that this hasn't been tested on more than a few
username/password pairs, there might be other issues lurking.

[Danish] Open source og LoRa er lidt vel store ord
Jan 6th, 2017 by miki

Min kommentar til LoRa og LoRaWANs åbenhed i forbindelse med artikel i Version2 med overskriften “TDC holder fast i proprietær IoT-standard – andre kører billig open source”, som nok mest minder om et bidrag til “open washing” af LoRa/LoRaWAN-teknologien. Se mere om open washing i denne blogpost fra Open Knowledge Foundation Denmark (Engelsk udgave).

Det er noget misvisende at snakke om at LoRa eller LoRaWAN er “open source”. Man er i hvert fald nødt til at skelne hårdt mellem LoRa og LoRaWAN.

LoRa specificerer radiomodulationen i luften, og det er en proprietær teknologi udviklet af virksomheden Semtech, som både har varemærkebeskyttet og patenteret den (EPO-patent);

LoRa is a proprietary spread spectrum modulation scheme that is derivative of Chirp Spread Spectrum modulation  (CSS)

Kilde: Semtech AN1200.22, LoRa(TM) Modulation Basics (, afsnit 1)

LoRaWAN er en åben protokol der anvender LoRa som transmissionsmedie, men som yderligere definerer hvordan flere enheder der alle kan snakke LoRa kan fungere sammen i et netværk. Den specificeres af et samarbejde mellem mange interessenter i LoRa Alliance, herunder også Semtech.

Q: What is LoRaWAN(TM)?
A: The LoRa modulation is the pshysical layer (PHY), and LoRaWAN is a MAC protocol for a high capacity, long range star network that the LoRa Alliance is standardizing for Low Power WideArea Networks (LPWAN).

Kilde: Semtech LoRa® FAQ (, spgm. 3)

En mere teknisk beskrivelse kan findes her; LoRaWAN-specifikationen selv er næsten frit tilgængelig, det er kun folk med offentlige email-adresser der ikke må få den.

For mig ser det dog ud som om Semtech og LoRa Alliance helt bevidst mudrer sondringen mellem LoRa og LoRaWAN, og synes det lugter af at det udelukkende er Semtech der kan og må producere silicium der implementerer LoRa. Selvom jeg ikke har kunnet finde steder det bliver sagt helt eksplicit.

Microchip og andre har produkter der snakker LoRa, men Semtech ser ud til altid at være med inde over, så jeg vil tro det stadig er en chip fra deres fabrik der ligger til grund for LoRa-funktionaliteten.

Der er dog folk der har kigget i sprækkerne på LoRa;

Spændende hvad der sker på dette felt. En mere fri og uafhængig infrastruktur i åbne licensbånd skal da være så velkommen.


»  Substance:WordPress   »  Style:Ahren Ahimsa
© 2016 Mikini Services