SIDEBAR
»
S
I
D
E
B
A
R
«
HOWTO: restore an iPad using only Free Software
Feb 14th, 2018 by miki

Thanks to the fine people at the libimobiledevice project, who bothers to reverse engineer Apple products, I recently succeeded in resurrecting a relative’s iPad stuck in a boot loop (something with jailbreaking, running Sydia, missing an iOS update and attempted Sydia removal) without any use of proprietary tools.

This is a brief recipe of the procedure done using Ubuntu 16.04.

As the required tool from libimobiledevice, idevicerestore, is not packaged in the Ubuntu libimobiledevice package we need to build this from scratch from the sources.

iPad during recovery

iPad in recovery mode during firmware download using libimobiledevice

  1. Install build dependencies
    sudo apt install libusbmuxd-dev libplist-dev libplist++-dev libzip-dev
  2. fetch and build libimobiledevice main library
    cd
    git clone https://git.libimobiledevice.org/libimobiledevice.git
    cd libimobiledevice/
    ./autogen.sh
    make
  3. fetch and build libirecovery library
    cd
    git clone https://git.libimobiledevice.org/libirecovery.git
    cd libirecovery
    ./autogen.sh
    make
  4. fetch and build idevicerestore tool, using the homebuilt libraries
    cd
    git clone https://git.libimobiledevice.org/idevicerestore.git
    cd idevicerestore
    CFLAGS="-I$HOME/libirecovery/include -I$HOME/libimobiledevice/include" LDFLAGS="-L$HOME/libirecovery/src/.libs -L$HOME/libimobiledevice/src/.libs" PKG_CONFIG_PATH=~/libirecovery:~/libimobiledevice/src ./autogen.sh
    make
  5. put the iDevice in recovery mode (iPad = press power+home until screen with “iTunes+cable” symbol appear, see image above and check Apple support for details), make sure it has adequate charge or it will refuse (red battery flashing)
  6. perform the actual restore, asking for flashing of latest firmware (~2.5GiB automatically downloaded), this will probably get you in trouble if you desire to jailbreak the device. I noticed while writing this post that the below actually doesn’t run the tool using the libraries built above, but I’m leaving it as it was done because it “worked for me” (TM) and I can’t experiment further because I haven’t got access to any iDevices (and desire to keep it that way):
    sudo $HOME/idevicerestore/src/idevicerestore --latest
    NOTE: using cached version data
    Found device in Recovery mode
    Identified device as j71ap, iPad4,1
    Latest firmware is iPad_64bit_11.2_15C114_Restore.ipsw
    Verifying 'iPad_64bit_11.2_15C114_Restore.ipsw'...
    Checksum matches.
    Extracting BuildManifest from IPSW
    Product Version: 11.2
    Product Build: 15C114 Major: 15
    INFO: device serial number is DMPM4V3SFK15
    Device supports Image4: true
    Variant: Customer Upgrade Install (IPSW)
    This restore will update your device without losing data.
    Using cached filesystem from 'iPad_64bit_11.2_15C114_Restore/058-86080-124.dmg'
    Found ECID 6653578882512
    Getting ApNonce in recovery mode... 03 6b cc ac 57 8a b4 29 29 c1 a9 fe e4 97 54 3b a8 36 59 5a 
    Trying to fetch new SHSH blob
    Getting SepNonce in recovery mode... df 5c ad 67 48 bd 38 b4 6f 72 0a 5c b0 81 87 c3 95 37 4a da 
    WARNING: Unable to find BbChipID node
    WARNING: Unable to find BbSkeyId node
    Request URL set to https://gs.apple.com/TSS/controller?action=2
    Sending TSS request attempt 1... response successfully received
    Received SHSH blobs
    Extracting iBEC.ipad4.RELEASE.im4p...
    Personalizing IMG4 component iBEC...
    Sending iBEC (710360 bytes)...
    Recovery Mode Environment:
    iBoot build-version=iBoot-4076.30.43
    iBoot build-style=RELEASE
    Sending AppleLogo...
    Extracting applelogo@2x~ipad.im4p...
    Personalizing IMG4 component AppleLogo...
    Sending AppleLogo (22709 bytes)...
    ramdisk-size=0x10000000
    Extracting 058-85997-124.dmg...
    Personalizing IMG4 component RestoreRamDisk...
    Sending RestoreRamDisk (59978774 bytes)...
    Extracting DeviceTree.j71ap.im4p...
    Personalizing IMG4 component RestoreDeviceTree...
    Sending RestoreDeviceTree (101420 bytes)...
    Extracting kernelcache.release.ipad4...
    Personalizing IMG4 component RestoreKernelCache...
    Sending RestoreKernelCache (13226783 bytes)...
    About to restore device... 
    Waiting for device...
    Device 3fb0f5cc97b83c61c85d4b8333796d9e536a4c83 is now connected in restore mode...
    Connecting now...
    Connected to com.apple.mobile.restored, version 15
    Device 3fb0f5cc97b83c61c85d4b8333796d9e536a4c83 has successfully entered restore mode
    Hardware Information:
    BoardID: 16
    ChipID: 35168
    UniqueChipID: 6653578882512
    ProductionMode: true
    Starting FDR listener thread
    About to send NORData...
    Found firmware path Firmware/all_flash
    Getting firmware manifest from build identity
    Extracting LLB.ipad4.RELEASE.im4p...
    Personalizing IMG4 component LLB...
    Extracting applelogo@2x~ipad.im4p...
    Personalizing IMG4 component AppleLogo...
    Extracting batterycharging0@2x~ipad.im4p...
    Personalizing IMG4 component BatteryCharging0...
    Extracting batterycharging1@2x~ipad.im4p...
    Personalizing IMG4 component BatteryCharging1...
    Extracting batteryfull@2x~ipad.im4p...
    Personalizing IMG4 component BatteryFull...
    Extracting batterylow0@2x~ipad.im4p...
    Personalizing IMG4 component BatteryLow0...
    Extracting batterylow1@2x~ipad.im4p...
    Personalizing IMG4 component BatteryLow1...
    Extracting glyphplugin@2x~ipad-lightning.im4p...
    Personalizing IMG4 component BatteryPlugin...
    Extracting DeviceTree.j71ap.im4p...
    Personalizing IMG4 component DeviceTree...
    Extracting recoverymode@2x~ipad-lightning.im4p...
    Personalizing IMG4 component RecoveryMode...
    Extracting iBoot.ipad4.RELEASE.im4p...
    Personalizing IMG4 component iBoot...
    Extracting sep-firmware.j71.RELEASE.im4p...
    Personalizing IMG4 component RestoreSEP...
    Extracting sep-firmware.j71.RELEASE.im4p...
    Personalizing IMG4 component SEP...
    Sending NORData now...
    Done sending NORData
    About to send RootTicket...
    Sending RootTicket now...
    Done sending RootTicket
    Waiting for NAND (28)
    Checking filesystems (15)
    Checking filesystems (15)
    Unmounting filesystems (29)
    Unmounting filesystems (29)
    Creating filesystem (12)
    About to send filesystem...
    Connected to ASR
    Validating the filesystem
    Filesystem validated
    Sending filesystem now...
    [==================================================] 100.0%
    Done sending filesystem
    Verifying restore (14)
    [==================================================] 100.0%
    Checking filesystems (15)
    Checking filesystems (15)
    Mounting filesystems (16)
    Mounting filesystems (16)
    About to send KernelCache...
    Extracting kernelcache.release.ipad4...
    Personalizing IMG4 component KernelCache...
    Sending KernelCache now...
    Done sending KernelCache
    Installing kernelcache (27)
    About to send DeviceTree...
    Extracting DeviceTree.j71ap.im4p...
    Personalizing IMG4 component DeviceTree...
    Sending DeviceTree now...
    Done sending DeviceTree
    Certifying Savage (61)
    Flashing firmware (18)
    [==================================================] 100.0%
    Updating gas gauge software (47)
    Updating gas gauge software (47)
    Updating Stockholm (55)
    About to send FUD data...
    Sending FUD data now...
    Done sending FUD data
    About to send FUD data...
    Sending FUD data now...
    Done sending FUD data
    Fixing up /var (17)
    Modifying persistent boot-args (25)
    Unmounting filesystems (29)
    Unmounting filesystems (29)
    Got status message
    Status: Restore Finished
    Cleaning up...
    DONE
  7. The iDevice should reset and boot into the new firmware.
iPad during firmware flashing using libimobiledevice

iPad during firmware flashing using libimobiledevice

If you want to interact with iDevices from within Ubuntu during ordinary use, you could also install some utils and plugins for that. Below will fx. add a context menu in nautilus with info about the iDevice and install the ideviceinstaller command line utility which can be used to administer installed applications on the device.

sudo apt install libimobiledevice-utils nautilus-ideviceinfo ideviceinstaller

Setting free a Google Authenticator TOTP secret
Nov 7th, 2017 by miki

Below you’ll find an illustrated guide for freeing the authentication mechanism used by the Google Authenticator app for Android or iPhone for use on your favorite device (anywhere an implementation is available). In fact the Authenticator is using a standards based 2FA (two factor authentication) scheme defined by OATH (Initiative for Open Authentication) and published in RFC6238 dubbed TOTP – Time-Based One-Time Password Algorithm (more Authenticator background and it’s basis HOTP).

This is a rather fine contraption but Google doesn’t advertise it very loudly being a standard instead locking the generated TOTP secret into a QR code that they will only imply are for use by their own Google Authenticator. That is not true as a host of alternative Android OTP apps are compatible and can read the QR codes as they are based upon the Authenticator’s legacy as an open source application which Google took private.

Retrieve Google Account QR Code

Login to your Google Account (maybe using the authenticator?) and go to Account -> Security -> Signin -> 2-step verification.

Locate the “Authenticator app” section and Click “CHANGE PHONE” (really “CHANGE TOTP SECRET”).

Screenshot from 2017-11-06 22-32-05

In my country’s locale, Danish, this is BTW mistranslated as “SKIFT TELEFONNUMMER” = change phone number);

Screenshot from 2017-11-07 00-50-06

You’ll now get a choice between either iPhone or Android, this will only affect the link to the app store shown on next screen which also contains the QR code, the one we are really after:

Screenshot from 2017-11-06 23-03-44

Right click the QR code image, select to save it and put it somewhere you can find it (~/Google_TOTP_QR.png might be sensible).

Get your tools right

Install the QR-code decoder zbar for extracting the TOTP secet from the image and the OATH toolkit oathtool for generating future TOTP passwords using it. On Debian/Ubuntu this can be done by installing the packages zbar-tools and oathtool.

 

$ sudo apt install zbar-tools oathtool

 

Now extract the otpauth URI (seems to be a Google thing) by passing the image file to zbarimg;

 

$ zbarimg ~/Google_TOTP_QR.png
 QR-Code:otpauth://totp/Google%3Auser%40domain.com?secret=pca7uyfht7f6mfs7oiec4aeavxaevish&issuer=Google
 scanned 1 barcode symbols from 1 images in 0.02 seconds

 

The URI contains all parameters to input into the TOTP algorithm for generating a password usable for 2FA authentication, notably the secret key in base32 format. Defaults are not mentioned in the URI and are not necessary to specify explicitly for oathtool. So for the above URI specifying only a secret, a password can be generated as such;

$ oathtool --totp --base32 pca7uyfht7f6mfs7oiec4aeavxaevish
 491293

“491293” being the password, having a default validity of 30 seconds (called step size because of the counter based nature of the HOTP behind it).

If your are interested in the defaults assumed by oathtool they can be viewed by supplying -v to it (missing );

$ oathtool --totp --base32 pca7uyfht7f6mfs7oiec4aeavxaevish -v
Hex secret: 7881fa60a79fcbe6165f72082e0080adc04aa247
Base32 secret: PCA7UYFHT7F6MFS7OIEC4AEAVXAEVISH
Digits: 6
Window size: 0
Step size (seconds): 30
Start time: 1970-01-01 00:00:00 UTC (0)
Current time: 2017-11-06 23:23:31 UTC (1510010611)
Counter: 0x30007F7 (50333687)

133339

 

Use & Behold

Enjoy having cut your reliance on the Google Autenticator, and opened up for your own choice of client:

Also see the dongleauth.info website for a list of other sites supporting TOTP and U2F.

Clean Up & Secure

Be sure to delete the QR code image when finished, and seek out a suitably secure place to store your TOTP secret. One solution could be using PGP encryption, maybe even through some clever management system such as pass.

Also educate yourself of the shortcomings of TOTP/HOTP and consider studying and using the FIDO U2F standard and related devices.

When email in a .msg file is not ASCII
Sep 18th, 2017 by miki

Got myself an exported message from (apparently) Exchange/Outlook in a file with .msg extension. My initial thought was that this was just a plain ASCII email (seem to remember having handled .msg files as such at some point), but looking at it as text exposed a load of binary and the “file” utility reported it being of type “Composite Document File V2 Document, Cannot read section info”.

What is it?

Some searching reveals .msg is in a proprietary format called “Outlook Item (.msg) File Format” (or formally, MS-OXMSG, find the specification  in the MSDN document entitled “[MS-OXMSG]: Outlook Item (.msg) File Format“). This format is related to the CFBF ([MS-CFB]: Compound File Binary File Format).

Check out what Library of Congress has to say about .msg files.

Interpreting it

libgsf

The GNOME project has developed an open source library and some tools for interacting with files of this type, check out libgsf documentation at developer.gnome.org/gsf/ and source code at github.com/GNOME/libgsf. Or install the package named libgsf-bin in Debian/Ubuntu for access to the “gsf” binary (man page) that can inspect and dump contents from a .msg file. This is quite rudimentary if you just want to read the content.

MSGViewer (java)

MSGViewer is a stand alone java application that can display/convert different mail-format, including .msg. On GitHub I found a fork which claims to provide further features than the original.

msgconv (perl)

msgconv (github source) is a perl script that converts a .msg file into standard RFC822/2822/5322 format .eml (email) format. Install as package “libemail-outlook-message-perl” on Debian/Ubuntu.

$ file test.msg
test.msg: Composite Document File V2 Document, Cannot read section info
$ msgconvert test.msg
$ file test.eml
test.eml: UTF-8 Unicode text, with very long lines, with CRLF line terminators

I prefer the approach of using a standard format for storing data, so I ended up accepting the .eml from msgconv which I propagated to other recipients of the file.

ToS humor: Tagalog speakers at Meetup.com
Jul 4th, 2017 by miki

A little humor is possible even when filtered through the eyes of lawyers. I’ll try to make a write-up on the ones I stumble across when reading ToS’es in the future.

The first is from the event site meetup.com which has embedded a sentence in the Tagalog language in the “Translation” section of their most recent edition of the site’s Terms of Service agreement (I was prompted to agree today even though the last update was on March 28, more than 3 months ago).

The text is “Umaasa kami na maintindihan mo” which according to Google Translate translated to “We hope you understand it“.

Even though they provide the previous editions for reference (it ought be illegal not to) I haven’t yet agreed to it as there is not an easy way to compare, it is a rather long read and there are non-trivial changes. I’d rather not use any service than to just agree carelessly to ToS updates (no, that is not a personality flaw).

 

Teknologi-danskere: Finn Hendil – Prøvebilledet
Apr 6th, 2017 by miki

Hvis du har mere end 15 leveår på bagen, har du nok set dette billede før:

Farveprøvebillede fra Phillips PM5544

Måske ved du også at en dansker har designet dette tidligere så berømmede prøvebillede? Det var ukendt for mig, før jeg læste dette Facebook opslag fra Universe Science Park (tidligere Danfoss Universe).

Men den danske ingeniør Finn Hendil, som arbejdede for Phillips i Danmark, var ham der i 1966 konstruerede grundlaget for dette mønster til test af farvefjernsyn, som i de følgende mange år blev solgt i apparaterne PM5540 og PM5544. De anvendtes til generering af prøvebilleder til udsendelse direkte fra TV-kanaler og som testudstyr hos fjernsynsproducenter.

Historien som fortalt af Erling Madsen i publikationen “Fjernsynets historie fra 1926 til 2012 – og lidt efter …“.

I Jenagade på Amager havde Philips en udviklingsafdeling og en produktion, som siden 1930’erne havde skiftet mellem bl.a. radioer, dyremad (under 2. verdenskrig), barbermaskiner, kanal-
vælgere til fjernsyn og radionavigation til lystsejlere samt mobiltelefoner. Ingeniør Finn Hendil ledede udviklingsafdelingen, og i 1966 fik han opgaven med at lave et prøvebillede, såteknikerne kunne se om farverne på farvefjernsynet stod rigtigt. Det blev siden dét farveprøvebille de, som blev standarden verden over og stadig bruges. Det er endda også udkommet på puder og t-shirt – og det var med i dødsannoncen, da Finn Hendil døde i januar 2011; en måned før han ville være fyldt 72 år.

 

Monokromt prøvebillede fra forløberen Phillips PM5540

Stallman in 2012: Denmark supposedly a free country, still valid
Mar 3rd, 2017 by miki

Stumbled upon this slightly dated talk by Richard M. Stallman (aka. RMS) of GNU and FSF fame, in which my home country of Denmark is sadly referenced as only a “supposedly free country”.

Transcript

“But censorship is wrong, of course, whether it is done on the internet or not. We used to think that the internet would protect us from censorship because it was too hard to censor the internet. But thanks to the effort of various companies in the US, The UK, France and so on, it is now possible for governments to censor the internet and also surveil it completely, they just need to put enough effort in. And this is not limited to obvious tyrannies such as China and Iran. We see a lot of supposedly free countries imposing censorship on the internet.

For instance, Denmark several years ago imposed filtering on the internet blocking a secret list of sites. The list was leaked and posted on WikiLeaks. Hooray for WikiLeaks! Whereupon Denmark blocked access to that page too. So everyone else could know what internet users in Denmark were blocked from seeing except those people.”

Sadly since this time it has not gotten any better. Most of the points RMS makes (the whole talk is worth a listen) are still valid and a grave concern from my perspective. The Danish internet (really DNS) blocking system has been broadened and the slippage that was feared has become a reality. Even though this issue has gotten some attention in the IT and rights communities the general public just doesn’t care.

The actual block is technically done through DNS blacklists that Danish ISP are legally required to implement. The list of blocked sites is available from the telecom trade organization “Telekommunikationsindustrien i Danmark” (English: Telecommunication’s Industry Association in Denmark) at teleindu.dk/brancheholdninger/blokeringer-pa-nettet/ and currently has 111 sites (csv) on active block.

As it being DNS based if you are impacted, workarounds do exist. However, my guess is that they will soon be able to actively shut down services physically located in Denmark.

Full talk

(starting at point of above transcript)

 

Generating passwords for Mosquitto MQTT broker using PHP
Jan 13th, 2017 by miki

Here is a delayed writeup of a my involvement in a question posted to the liberally licensed MQTT broker (server) Mosquitto’s developer list list about how to generate authentication tokens programmatically. It kicked the curious cat in me which propelled a journey into the backyards of the C code for the mosquitto_password tool which normally is used for this purpose. This resulted in the proof of concept PHP implementation outlined in my answer on the list which is reproduced below.

MQTT (once Message Queue Telemetry Transport) is a lightweight publish/subscribe protocol intended for communication between low power, low bandwidth embedded devices. These days it is commonly hyped as a holy grail in the religion of IoT. The protocol was originally developed by IBM but is now a standard overseen by the OASIS standardization organization which also has the OpenDocument standard (ODF, think Open/Libre-Office) under its wings. According to Wikipedia MQTT is used behind the scenes of Facebook Messenger, OpenStack and Amazon’s IoT services.

For further practical use of the concept outlined you would need to produce a random 16 byte base64 encoded salt to feed into the hasher, that could be done using something like; $salt_base64=base64_encode(openssl_random_pseudo_bytes(12));

If your need a shrink wrapped solution to this you could try to ping me.

Hi Srinivas.

On 2016-07-26 12:49, Srinivas Pokala wrote:
> Username successfully created using linux command with: 
> "mosquitto_passwd /etc/mosquitto/passwd guest".
> I need to create same with php or javascript how?

Looking at the source of mosquitto_passwd
(https://github.com/eclipse/mosquitto/blob/master/src/mosquitto_passwd.c)
basically all it does to generate the resulting line you see in the
password file is:

1) draw a random 12 byte binary salt
2) hash the combination of password and salt using sha512
3) write username, base64 encoded salt, base64 encoded hash in one line

A PHP implementation would use something like this ($salt is fixed for
demonstration purposes, it ought to be random in production);

---
$username="Bitten";
$password="Insect";
$salt_base64="spicychilinstuff";
$salt=base64_decode($salt_base64);
$hash=hash("sha512",$password.$salt, true);
$hash_base64=base64_encode($hash);
echo($username.":$6$".$salt_base64."$".$hash_base64."\n");
---

Comparing against mosquitto_passwd using a one-liner (uses the base64
salt from output to be able to correlate the two);

---
$ mosquitto_passwd -b ~/mosq_passwd_test Bitten Insect
$ cat ~/mosq_passwd_test
Bitten:$6$mfJ0Eq3rIDLKG33r$gkiIlz80JA6Pq9OtGhasIsx7L2vf0APdZH77+thmNW2Zp5vE1d/dAi5TjbfO9mZpKHLh38Oem1ic072rSO328g==

$ php -r '$username="Bitten"; $password="Insect";
$salt_base64="mfJ0Eq3rIDLKG33r"; $salt=base64_decode($salt_base64);
$hash=hash("sha512",$password.$salt, true);
$hash_base64=base64_encode($hash);
echo($username.":$6$".$salt_base64."$".$hash_base64."\n");'
Bitten:$6$mfJ0Eq3rIDLKG33r$gkiIlz80JA6Pq9OtGhasIsx7L2vf0APdZH77+thmNW2Zp5vE1d/dAi5TjbfO9mZpKHLh38Oem1ic072rSO328g==
---

As can be seen, the PHP generated password line are identical to the
mosquitto_passwd generated.

I have also successfully tested authentication against the mosquitto
broker with PHP generated users. One caveat is that the above can
generate a salt of arbitrary length, but the broker must see a 12 byte
binary salt (16 byte base64) or authentication will fail.

Note however, that this hasn't been tested on more than a few
username/password pairs, there might be other issues lurking.

Regards,
-- 
Mikkel
[Danish] Open source og LoRa er lidt vel store ord
Jan 6th, 2017 by miki

Min kommentar til LoRa og LoRaWANs åbenhed i forbindelse med artikel i Version2 med overskriften “TDC holder fast i proprietær IoT-standard – andre kører billig open source”, som nok mest minder om et bidrag til “open washing” af LoRa/LoRaWAN-teknologien. Se mere om open washing i denne blogpost fra Open Knowledge Foundation Denmark (Engelsk udgave).

Det er noget misvisende at snakke om at LoRa eller LoRaWAN er “open source”. Man er i hvert fald nødt til at skelne hårdt mellem LoRa og LoRaWAN.

LoRa specificerer radiomodulationen i luften, og det er en proprietær teknologi udviklet af virksomheden Semtech, som både har varemærkebeskyttet og patenteret den (EPO-patent);

LoRa is a proprietary spread spectrum modulation scheme that is derivative of Chirp Spread Spectrum modulation  (CSS)

Kilde: Semtech AN1200.22, LoRa(TM) Modulation Basics (http://www.semtech.com/images/datasheet/an1200.22.pdf, afsnit 1)

LoRaWAN er en åben protokol der anvender LoRa som transmissionsmedie, men som yderligere definerer hvordan flere enheder der alle kan snakke LoRa kan fungere sammen i et netværk. Den specificeres af et samarbejde mellem mange interessenter i LoRa Alliance, herunder også Semtech.

Q: What is LoRaWAN(TM)?
A: The LoRa modulation is the pshysical layer (PHY), and LoRaWAN is a MAC protocol for a high capacity, long range star network that the LoRa Alliance is standardizing for Low Power WideArea Networks (LPWAN).

Kilde: Semtech LoRa® FAQ (http://www.semtech.com/wireless-rf/lora/LoRa-FAQs.pdf, spgm. 3)

En mere teknisk beskrivelse kan findes her; https://www.lora-alliance.org/What-Is-LoRa/Technology. LoRaWAN-specifikationen selv er næsten frit tilgængelig, det er kun folk med offentlige email-adresser der ikke må få den.

For mig ser det dog ud som om Semtech og LoRa Alliance helt bevidst mudrer sondringen mellem LoRa og LoRaWAN, og synes det lugter af at det udelukkende er Semtech der kan og må producere silicium der implementerer LoRa. Selvom jeg ikke har kunnet finde steder det bliver sagt helt eksplicit.

Microchip og andre har produkter der snakker LoRa, men Semtech ser ud til altid at være med inde over, så jeg vil tro det stadig er en chip fra deres fabrik der ligger til grund for LoRa-funktionaliteten.

Der er dog folk der har kigget i sprækkerne på LoRa;

Spændende hvad der sker på dette felt. En mere fri og uafhængig infrastruktur i åbne licensbånd skal da være så velkommen.

Mikkel

[Danish] S&S: gemme data i Arduino ROM/Flash (PROGMEM / F())
Dec 21st, 2016 by miki

Mit svar på et spørgsmål i Facebook-gruppen Danske Arduino Entusiaster omkring Arduino ROM/Flash, PROGMEM og system-inklude-filer.

Spørgsmål

Hej er der en der ved hvor jeg kan hente dett lib. <avr/pgmspace.h> jeg skal bruge denne funktion PROGMEM
så jeg kan gemme et billede i Arduino uden SD kort
det kan være der er en der kender en anden måde at gøre det på.

Svar

pgmspace.h er en inklude-fil som er en del af c-biblioteket til AVR-arkitekturen (avr-libc). C-bibliotekets inklude-filer vil normalt ligge i kompilerens “system include”-sti (se GCC options -I og -isystem). Dermed kan den inkluderes blot med “#include <avr/pgmspace.h>”. Se evt. også Arduino-referencen på https://www.arduino.cc/en/Reference/PROGMEM.
 
Bemærk at PROGMEM ikke er en funktion, men en storage modifier (lager-modifikator) som fortæller kompileren at den kan placere en en given variabel i ikke-skrivbar lager (ROM/Flash). Der skal efterfølgende anvendes specielle funktioner til at læse data fra en sådan variabel (se referencen).
Arduino-frameworket har dog lavet en nem måde at placere konstant-strenge i Flash på (normalt lagres de i SRAM!), nemlig funktionen F() som kan anvendes direkte i f.eks. printf/write/print (Serial.print(F(“Waiting for connection”));)
 
Hvis du vil inspicere indholdet af pgmspace.h, kan du finde filen i Arduino IDE’ets installations-mappe under hardware/tools/avr/avr/include/avr/pgmspace.h. Det er ikke en man kan/skal redigere manuelt i, da den er tæt koblet med den binære kode i selve biblioteket.
 
Der findes også EEPROM-lager du sikkert vil kunne bruge til samme formål; https://www.arduino.cc/en/Reference/EEPROM

Se svaret på Facebook.

Den videre færd med F()

Da jeg ikke kunne finde en uddybende forklaring på F()-funktionen (som egentlig er en makro) i Arduino-dokumentationen (brugen nævnes meget kort i PROGMEM , Memory og Print), gravede jeg efterfølgende lidt rundt for at lære mere. I de sparsomme Arduino-eksempler er den anvendt udelukkende med konstante strenge, hvilket også viser sig at være et krav (eller i hvert fald noget der kan castes til const char *).

Makroen er defineret af Arduino-frameworket i filen hardware/arduino/avr/cores/arduino/WString.h (referencerne er ifht. min lokale installation af Arduino 1.6.9, pt. er nyeste 1.6.13) således:

#define F(string_literal) (reinterpret_cast<const __FlashStringHelper *>(PSTR(string_literal)))

Altså parametren til F() bruges som parameter til PSTR() (progmem string, er mit bud på navn) som er en makro defineret i pgmspace.h fra avr-libc.

Dens funktion er at caste parametrens type til konstant streng-pointer med PROGMEM modifier;

#define PSTR(s) ((const PROGMEM char *)(s))

Skal vi se på hvad PROGMEM rent faktisk er, så finder vi endnu et sæt makroer der ender med at blive udviddet til kompiler-attributten  __progmem__, igen definieret i pgmspace.h (hardware/tools/avr/avr/include/avr/pgmspace.h):

#define PROGMEM __ATTR_PROGMEM__

#define __ATTR_PROGMEM__ __attribute__((__progmem__))

__progmem__ attributten er en instruks til kompileren (GCC) og linkeren om ved programmering/flashing af programmet at placere disse data i en sektion af hukommelsen der hedder “.progmem“. Se evt. mere om dette i GCC-kompilerens dokumentation. For hver AVR-chip kompileren understøtter er der eksakte definitioner af hvilke hukommelsesadresser .progmem ligger på for netop denne chip.

Dvs. når man i sin kode skriver F(“test”) får man i virkeligheden:

(reinterpret_cast<const __FlashStringHelper *>(((const __attribute__((__progmem__)) char *)(“test”)))

Altså en konstant streng der lagres i AVR-processorens progmem-sektion, og som returværdi får en pointer til en konstant instans af en klasse kaldet “__FlashStringHelper“. Denne klasse må være lavet sådan at den anvender de korrekte mekanismer til at læse fra progmem-området (måske mere om dette i en senere artikel). Arduinos funktion-bibliotek (Serial.print() mm.) er lavet således at de direkte kan tage en parameter af denne type som erstatning for en konstant-streng (og det er netop her Arduino-frameworket viser sin værdi ved at abstrahere sådanne kompleksiteter væk fra programmøren).

One-liner: Decompress file and diff against another file
Dec 8th, 2016 by miki

Below is a handy shell one-liner for comparing the decompressed contents of a compressed file against a plain file. The purpose here is to test whether the compressed file is actually derived from compressing the plain file. This particular example is from a real life situation where log rotation by cron.daily on a Ubuntu server had begun failing. The situation is thought to be the result of an interrupted logrotate execution leaving a compressed (but non-rotated) intermediate file in the file system that prevented further log rotation on subsequent executions.

The command constructs a pipe between two process, specified using the | symbol (vertical line), to send the decompressed contents from gunzip stdout to stdin of the compare tool. The example uses diff as compare tool which is suitable for textual contents. You could use f.x. cmp instead if the contents is binary. Both diff and cmp interprets the file name “-” as meaning that input should be read from stdin.

$ gunzip --stdout /var/log/syslog.1.gz | diff --report-identical-files - /var/log/syslog.1
Files - and /var/log/syslog.1 are identical

The above uses long options for clarity, in a real life situation you would probably be using short options instead. That means -c instead of –stdout and s instead of –report-identical-files.

$ gunzip -c /var/log/syslog.1.gz | diff -s - /var/log/syslog.1
Files - and /var/log/syslog.1 are identical

 

»  Substance:WordPress   »  Style:Ahren Ahimsa
© 2016 Mikini Services