SIDEBAR
»
S
I
D
E
B
A
R
«
Beaglebone Black periodic boot failure; patching mainline u-boot
Jan 15th, 2015 by miki

Patch for u-boot mainline master (http://git.denx.de/u-boot.git) to prevent BBB’s to get stuck in a u-boot prompt because of spurious characters being received on the serial console (see http://www.mikini.dk/index.php/category/beaglebone-black/boot-issue).

diff –git a/include/configs/ti_am335x_common.h b/include/configs/ti_am335x_common.h
index 5ed86d9..c58f467 100644
— a/include/configs/ti_am335x_common.h
+++ b/include/configs/ti_am335x_common.h
@@ -12,6 +12,12 @@
#ifndef __CONFIG_TI_AM335X_COMMON_H__
#define __CONFIG_TI_AM335X_COMMON_H__

+#define CONFIG_AUTOBOOT_KEYED
+#define CONFIG_AUTOBOOT_STOP_STR “stop”
+#define CONFIG_AUTOBOOT_PROMPT “autoboot in %d seconds (type ‘%s’ to abort)\n”,bootdelay,CONFIG_AUTOBOOT_STOP_STR
+#define CONFIG_BOOT_RETRY_TIME 30
+#define CONFIG_RESET_TO_RETRY
+
#define CONFIG_AM33XX
#define CONFIG_ARCH_CPU_INIT
#define CONFIG_SYS_CACHELINE_SIZE       64
@@ -102,4 +108,7 @@
/* Now bring in the rest of the common code. */
#include <configs/ti_armv7_common.h>

+#undef  CONFIG_BOOTDELAY
+#define CONFIG_BOOTDELAY               5
+
#endif /* __CONFIG_TI_AM335X_COMMON_H__ */

Patch and compiled binaries at http://www.mikini.dk/wp-content/uploads/2015/01/u-boot_mainline_BBB-autoboot-patch_201501151.zip.

Install the new u-boot by copying the files “MLO” and “u-boot.img” to the root directory of your boot device (first FAT-partition on your SD-card or onboard MMC). Using the stock Debian image (http://beagleboard.org/latest-images) this can be done via USB by powering the board from your computers USB-interface, waiting for the BBB to boot and register its drive as an usb mass-storage in your OS. Now use your favorite file management application to copy the files from the above zip-file replacing the existing files.

Disclaimer: this is mostly an experiment, there is a lot of u-boot trees and patches floating around for the BBB (like https://github.com/beagleboard/u-boot), so probably mainline hasn’t got the most recent stuff for AM335x/BBB yet.

Beaglebone Black periodic boot failure; no high required, just stable voltage
Nov 6th, 2014 by miki

This is test report 3 in the series of tests on the “BBB doesn’t boot” issue, discussed here on the BBB mailing list.

The present test is accompanying this specific post in the discussion thread.

The goal of the test is to establish under which conditions the U15.2 (1A)  input provides a stable boot experience. The four test subjects are a strong pull down of 990 ohm and 0 ohm, and voltage divider circuits using 0 ohm and 1k ohm fixing the voltage at respectively 3.3V and 0.58V.

Results

The strong pull down of 990 ohm and 0 ohm on B_UART0_RX doesn’t prove successful, as was also the case with the weaker pull down of 45k2 ohm in test report 2, and the factory mounted pull down of 100k ohm.

But providing a stable 3.3V or 0.58V using a voltage divider with resistor values 0 ohm and 1k ohm results in a booting BBB in every test case!

This is analogous with the result of test 2 in test report 2, which established the same fact, but for B_UART0_RX = 1.81V using a 82k5 ohm resistor.

The second picture below shows an easy and safe way to establish the condition of test 3 as a permanent fix on the backside of the BBB pcb. It places an insulated wire between VDD_3V3B from terminal 5 on the non-populated P2″CTI JTAG, DNI” header and the B_UART0_RX signal on J1 (UART0 Serial Port) pin 4.

Failure Rates

  • Pull down of 990 ohm: 2 fails/60  boots= 3.33%
  • Forced 0V: 3 fails/60  boots= 5.00%
  • Forced 3.3V: 0 fails/100 boots= 0.00%
  • Forced 0.58V: 0 fails/60  boots= 0.00%

Pictures

Detailed Test Report

(formatted in nice emacs org-mode)

* BBB boot lockup test report 3

** Equipment
*** Device Under Test
    Beaglebone Black (BBB) produced by Element 14 (PCB REV B6, serial EM-400524+XA6001961,
    marked "Element 14").

*** Device Under Test #1
    Modify DUT by applying an additional 1k ohm pull down resistor in parallel to R165
    from J1.4 (B_UART0_RX)/U15.2 (1A) to P8.1 (DGND), thus forming a very strong pull 
    down on B_UART0_RX with resistive value of 1/(1/1k+1/82k5)= 990.1 ohm.

*** Device Under Test #2
    Modify DUT by applying a short circuit from from J1.4 (B_UART0_RX)/U15.2 (1A) to
    P8.1 (DGND), thus forcing 0V on B_UART0_RX.

*** Device Under Test #3
    Modify DUT by applying a short circuit from J1.4 (B_UART0_RX)/U15.2 (1A) to
    P8.4 (VDD_3V3B), thus forcing 3.3V on B_UART0_RX.

*** Device Under Test #4
    Modify DUT by applying a 470k ohm pull up resistor from J1, pin 4 (B_UART0_RX,
    U15-pin 2, signal 1A) to P8, pin 4 (VDD_3V3B), effectively creating a voltage
    divider with existing pull down resistor R165 (100k ohm) fixing voltage on
    B_UART0_RX to 3.3V*100k/(470k+100k)= 0.58V.

*** Power Supply Unit
    Huawei HW-050200E3W, output 5V 2A, USB A-connector. Danish plug.
    Sourced from Huawei E589 mobile wifi.

*** Power Cable
    20 cm no-name USB A male connector to USB Mini-B male connector.

** Test 1+2+3+4 Procedure
   For test 1 use DUT#1, for test 2 use DUT#2, for test 3 use DUT#3, for test 4
   use DUT#4.
   Connect cable Mini-B male to DUT USB Mini-B female. Insert PSU into mains socket.

   Test boot capability of DUT by inserting the cable's USB A connector into the
   PSU while keeping the USB Mini-B connector inserted into the DUT. Then verify
   that the power led (D1) light up, and note whether boot succeeded or failed by
   watching if USR0-USR3 leds (D2-D5) lights up indicating boot. Then remove the
   A connector from the mains adaptor inserting it immediately repeating the test.

   Results can be seen in section Test Results.

** Interpretation

   Test 1 failure rate= 2 fails/60  boots= *3.33%*
   Test 2 failure rate= 3 fails/60  boots= *5.00%*
   Test 3 failure rate= 0 fails/100 boots= *0.00%*
   Test 4 failure rate= 0 fails/60  boots= *0.00%*

   The tests 1 & 2 shows that forming first a strong pull down (replacing 100k
   with 9k1) and then a short forcing 0V on B_UART0_RX, doesn't prevent the failure
   to occur.

   Whereas test 3+4 shows that forming a voltage divider which fixes the voltage
   instead of just pulling up/down indeed makes the system boot at every power up.

   Overall this indicates that the flickering of U15's output 1Y could be caused
   internally in U15 by a spurious input signal on input 1A during power up. This
   can't be elleviated by inserting pull-up/downs, but creating a stable input
   signal on 1A by a voltage divider does solvs the boot issue, disregarding
   whether this voltage is low (0.58V) or high (3.3V).

** Test results

| Boot no. | Test 1  | Test 2   | Test 3 | Test 4  |
|        1 | boot    | no boot  | boot   | boot    |
|        2 | boot    | boot     | boot   | boot    |
|        3 | boot    | boot     | boot   | boot    |
|        4 | boot    | boot     | boot   | boot    |
|        5 | boot    | boot     | boot   | boot    |
|        6 | boot    | boot     | boot   | boot    |
|        7 | boot    | boot     | boot   | boot    |
|        8 | boot    | boot     | boot   | boot    |
|        9 | boot    | boot     | boot   | boot    |
|       10 | boot    | boot     | boot   | boot    |
|       11 | boot    | boot     | boot   | boot    |
|       12 | boot    | boot     | boot   | boot    |
|       13 | boot    | boot     | boot   | boot    |
|       14 | boot    | boot     | boot   | boot    |
|       15 | boot    | boot     | boot   | boot    |
|       16 | boot    | boot     | boot   | boot    |
|       17 | boot    | no boot  | boot   | boot    |
|       18 | boot    | boot     | boot   | boot    |
|       19 | boot    | boot     | boot   | boot    |
|       20 | boot    | boot     | boot   | boot    |
|       21 | boot    | boot     | boot   | boot    |
|       22 | boot    | boot     | boot   | boot    |
|       23 | boot    | boot     | boot   | boot    |
|       24 | boot    | boot     | boot   | boot    |
|       25 | boot    | boot     | boot   | boot    |
|       26 | boot    | boot     | boot   | boot    |
|       27 | no boot | boot     | boot   | boot    |
|       28 | boot    | no boot  | boot   | boot    |
|       29 | boot    | boot     | boot   | boot    |
|       30 | boot    | boot     | boot   | boot    |
|       31 | boot    | boot     | boot   | boot    |
|       32 | boot    | boot     | boot   | boot    |
|       33 | boot    | boot     | boot   | boot    |
|       34 | boot    | boot     | boot   | boot    |
|       35 | boot    | boot     | boot   | boot    |
|       36 | boot    | boot     | boot   | boot    |
|       37 | boot    | boot     | boot   | boot    |
|       38 | no boot | boot     | boot   | boot    |
|       39 | boot    | boot     | boot   | boot    |
|       40 | boot    | boot     | boot   | boot    |
|       41 | boot    | boot     | boot   | boot    |
|       52 | boot    | boot     | boot   | boot    |
|       53 | boot    | boot     | boot   | boot    |
|       44 | boot    | boot     | boot   | boot    |
|       45 | boot    | boot     | boot   | boot    |
|       46 | boot    | boot     | boot   | boot    |
|       47 | boot    | boot     | boot   | boot    |
|       48 | boot    | boot     | boot   | boot    |
|       49 | boot    | boot     | boot   | boot    |
|       50 | boot    | boot     | boot   | boot    |
|       51 | boot    | boot     | boot   | boot    |
|       52 | boot    | boot     | boot   | boot    |
|       53 | boot    | boot     | boot   | boot    |
|       54 | boot    | boot     | boot   | boot    |
|       55 | boot    | boot     | boot   | boot    |
|       56 | boot    | boot     | boot   | boot    |
|       57 | boot    | boot     | boot   | boot    |
|       58 | boot    | boot     | boot   | boot    |
|       59 | boot    | boot     | boot   | boot    |
|       60 | boot    | boot     | boot   | boot    |
|       61 | ------- | -------- | boot   | ------- |
|       62 |         |          | boot   |         |
|       63 |         |          | boot   |         |
|       64 |         |          | boot   |         |
|       65 |         |          | boot   |         |
|       67 |         |          | boot   |         |
|       68 |         |          | boot   |         |
|       69 |         |          | boot   |         |
|       70 |         |          | boot   |         |
|       71 |         |          | boot   |         |
|       72 |         |          | boot   |         |
|       73 |         |          | boot   |         |
|       74 |         |          | boot   |         |
|       75 |         |          | boot   |         |
|       76 |         |          | boot   |         |
|       77 |         |          | boot   |         |
|       78 |         |          | boot   |         |
|       79 |         |          | boot   |         |
|       80 |         |          | boot   |         |
|       81 |         |          | boot   |         |
|       82 |         |          | boot   |         |
|       83 |         |          | boot   |         |
|       84 |         |          | boot   |         |
|       85 |         |          | boot   |         |
|       86 |         |          | boot   |         |
|       87 |         |          | boot   |         |
|       88 |         |          | boot   |         |
|       89 |         |          | boot   |         |
|       90 |         |          | boot   |         |
|       91 |         |          | boot   |         |
|       92 |         |          | boot   |         |
|       93 |         |          | boot   |         |
|       94 |         |          | boot   |         |
|       95 |         |          | boot   |         |
|       96 |         |          | boot   |         |
|       97 |         |          | boot   |         |
|       98 |         |          | boot   |         |
|       99 |         |          | boot   |         |
|      100 |         |          | boot   |         |
|          |         |          | ------ |         |
Beaglebone Black periodic boot failure; establishing failure rate and possible cause
Oct 31st, 2014 by miki

I’ve been hit by the “periodic boot failure” issue of the Beaglebone Black (aka BBB) reported by quite a few on the net. For most users this is an inconvenient annoyance, but for people, like me, using the platform in embedded applications, this issue causes a serious stability issue of the whole system, when 100% reliable boot is not achievable.

After having hunches about instability caused by the intermittent experiences during development, where the board was seen failing boot on power on, and not getting much more help from the net than “try the recommended power supply” (which btw. I can’t use because I live in a country where main sockets are non-US, even a bit non-European standard) I decided to make a systematic test to get the basic facts straight.

I settled on trying to establish some reasonable statistics about the error’s frequency on plain BBBs to have a reference against testing whether a theory put forward on the mailinglist (here and here) about the uboot bootloader being confused by noise being interpreted as valid data on UART0_RXD (pin E15) of the AM3358x  (see near U15 page 4, of the BBB REV B schematics) as the cause of the failure.

Results

This is the results (test report detailed below), I’m posting a writeup in the Beagleboard mailinglist (Edit: my post here), so hopefully you’ll find further discussion about this issue there soon.

Failure rates

Plain BBB (DUT#2+3), Element14 & mbest branded

  • DUT#2: Element14 branded: 4 fails/ 120 boots = 1/30 = 3.33%
  • DUT#3: mbest branded: 2 fail / 40 boots = 1/20 = 5.00
  • Overall: 6 fails / 160 boots = 3/80 = 3.75%

Modified BBB (DUT#1), CircuitCo branded

  • U15 removed, pull down on UART0_RXD : 0 fails / 40 boots = 0.00%
  • U15 removed, no pull down on UART0_RX: 0 fails / 40 boots = 0.00%
  • Overall: 0 fails / 80 boots = 0.00%

Interpretation

The two differing Element14 branded BBB products I have access to, but both PCB REV B6, exhibits a somewhat varying boot failure rate. But overall the boards fail to boot in almost 4 of 100 boots.

Investigating the theory relating to noise on UART0_RXD seems to have paid off, as first removing U15 (SN74LVC2G241: Dual Buffer/Driver With 3-State Outputs) for the purpose of adding a pull down on its pad 6 (which is connected to UART0_RXD)  alleviated the problem altogether. But also the experiment of removing the pull down and redo the test, showed that the act of removing U15 itself caused the boot to always succeed.

Unfortunately, in hindsight, I was too quick to grab the soldering iron, because I should have verified and quantized the occurrence of the failure on the actual board being modified. A shame It didn’t occur to me before modification, but I’ll be more than willing to try to remove U15 on DUT#2, which has had the highest failure rate, if discussions prove that it is a reasonable theory of the root cause of the failure. That is why I continued testing it through to 120 boots, to get more samples for improving statistics in the event that I pull U15 from it later.

Effect of removing U15

The success of removing U15 could be caused by the now floating AM3388 input UART0_RXD (pin E15) which presumably has a default weak internal pull up/down (the AM335x TRM says reset value is pad-dependent (register conf_uart0_rxd in Table 9.7 p. 1366 and  Section 9.3.1.50 p. 1420), which I haven’t yet figured out the exact meaning of)  stabilizing the signal

The activity when U15 is in place is somehow exhibited on output 1Y (pin 6) , probably because it is not stable, and thus has an erratic state, during the first moments of the chips power up sequence. This erratic behaviour can in fortunate/unfortunate circumstances be interpreted as valid bits and resulting bytes by the uart rxd cirtcuitry, which also can happen to be latched into the uart fifo rx buffer, waiting for uboot to read them when its code is executed looking for a user interrupt.

I’ll put in the disclaimer on this thesis, that I haven’t yet studied U15 in detail, but it is advertised as both a level converter, ESD protection and power live-insertion/partial-power-down suggesting it does something in reaction to it’s power condition.

Also the recommendation on page 1 of its datasheet; “To ensure the high-impedance state during power up or power down, OE (active low) should be tied to VCC through a pullup resistor, and OE (active high) should be tied to GND through a pulldown resistor”, seems not to be followed in the BBB circutry, as the OEs are hardwired in to be always active (opposite of the recommendation in the power up/down condition). If this is actually a problem, I need to do further analysis to establish.

Pictures

Detailed test report

(formatted in nice emacs org-mode)

* BBB boot lockup test report

** Device under test #1:
Modified Beaglebone Black produced by CircuitCo (PCB REV B6, serial 007142901445, marked
"beaglebone"+ beagle logo and "beagleboard.org").

Modified by adding hard pulldown resistor on TI AM3358 pin E15 (uart0 rx). Specifically
U15 was removed and terminal pin 6 (1Y=UART0_RX) was shorted to J1 pin 1 through a 82k5
ohm resistor.

** Device under test #2:
Unmodified Beaglebone Black (BBB) produced by Element 14 (PCB REV B6, serial
EM-400524+XA6001961, marked "Element 14").

** Device under test #3:
Unmodified Beaglebone Black (BBB) produced by mbest (PCB REV B6, serial
EM-400441+XA3001688).

** Power supply
Jentec Technologies CF1805-E, output 5V 3A. Danish plug.
Sourced from a D-Link DUB-H7 USB 2.0 HUB.

** Test 1 procedure

Each Beaglebone was tested by consequtively applying power by inserting the 
plug into the mains socket while keeping the DC barrel connector inserted and 
verifying that the power led light up, and then noting whether boot from SD-card 
succeeded or failed. Then removing the PSU from the mains connector waiting 5 
seconds and repeat.

The power supply and SD-card used was the same for all three DUTs.

Results can be seen in section Test 1 results.

** Test 2 procedure

After a short analysis of test 1 results I decided to try to remove the resistor, 
to see if the behavious was restored.

Otherwise test procedure was identical to test 1.
Results can be seen in section Test 2 results.

** Test 1 results (pull down and reference boards)

| Boot no | DUT#1 | DUT#2   | DUT#3   | Note                                  |
|       1 | boot  | boot    | boot    |                                       |
|       2 | boot  | boot    | boot    |                                       |
|       3 | boot  | boot    | boot    |                                       |
|       4 | boot  | boot    | boot    |                                       |
|       5 | boot  | boot    | boot    |                                       |
|       6 | boot  | boot    | boot    |                                       |
|       7 | boot  | boot    | boot    |                                       |
|       8 | boot  | boot    | boot    |                                       |
|       9 | boot  | boot    | boot    |                                       |
|      10 | boot  | boot    | boot    |                                       |
|      11 | boot  | no boot | boot    | DUT#2: pwr sw=lock, rst sw=boot       |
|      12 | boot  | boot    | boot    |                                       |
|      13 | boot  | boot    | boot    |                                       |
|      14 | boot  | boot    | boot    |                                       |
|      15 | boot  | boot    | boot    |                                       |
|      16 | boot  | boot    | boot    |                                       |
|      17 | boot  | boot    | boot    |                                       |
|      18 | boot  | boot    | boot    |                                       |
|      19 | boot  | boot    | boot    |                                       |
|      20 | boot  | no boot | boot    | DUT#2:  pwr sw=no boot, rst sw=boot   |
|      21 | boot  | boot    | boot    |                                       |
|      22 | boot  | boot    | boot    |                                       |
|      23 | boot  | boot    | boot    |                                       |
|      24 | boot  | boot    | boot    |                                       |
|      25 | boot  | boot    | boot    |                                       |
|      26 | boot  | boot    | boot    |                                       |
|      27 | boot  | boot    | boot    |                                       |
|      28 | boot  | boot    | boot    |                                       |
|      29 | boot  | boot    | boot    |                                       |
|      30 | boot  | boot    | boot    | DUT#3: pause before comencing test 31 |
|      31 | boot  | boot    | no boot | DUT#3: pwr sw=no boot, rst sw=boot    |
|      32 | boot  | no boot | boot    | DUT#2: pwr sw=no boot, rst sw=boot    |
|      33 | boot  | boot    | boot    |                                       |
|      34 | boot  | boot    | boot    |                                       |
|      35 | boot  | boot    | boot    |                                       |
|      36 | boot  | boot    | no boot | DUT#3: pwr sw=no boot, rst sw=boot    |
|      37 | boot  | boot    | boot    |                                       |
|      38 | boot  | boot    | boot    |                                       |
|      39 | boot  | boot    | boot    |                                       |
|      40 | boot  | boot    | boot    |                                       |
|      41 |       | boot    |         |                                       |
|      42 |       | boot    |         |                                       |
|      43 |       | boot    |         |                                       |
|      44 |       | boot    |         |                                       |
|      45 |       | boot    |         |                                       |
|      46 |       | boot    |         |                                       |
|      47 |       | boot    |         |                                       |
|      48 |       | boot    |         |                                       |
|      49 |       | boot    |         |                                       |
|      50 |       | boot    |         |                                       |
|      51 |       | boot    |         |                                       |
|      52 |       | boot    |         |                                       |
|      53 |       | boot    |         |                                       |
|      53 |       | boot    |         |                                       |
|      54 |       | boot    |         |                                       |
|      55 |       | boot    |         |                                       |
|      56 |       | boot    |         |                                       |
|      57 |       | boot    |         |                                       |
|      58 |       | boot    |         |                                       |
|      59 |       | boot    |         |                                       |
|      60 |       | boot    |         |                                       |
|      61 |       | boot    |         |                                       |
|      62 |       | boot    |         |                                       |
|      63 |       | boot    |         |                                       |
|      64 |       | boot    |         |                                       |
|      65 |       | boot    |         |                                       |
|      66 |       | boot    |         |                                       |
|      67 |       | boot    |         |                                       |
|      68 |       | boot    |         |                                       |
|      69 |       | boot    |         |                                       |
|      70 |       | boot    |         |                                       |
|      71 |       | boot    |         |                                       |
|      72 |       | boot    |         |                                       |
|      73 |       | boot    |         |                                       |
|      74 |       | boot    |         |                                       |
|      75 |       | boot    |         |                                       |
|      76 |       | boot    |         |                                       |
|      77 |       | boot    |         |                                       |
|      78 |       | boot    |         |                                       |
|      79 |       | boot    |         |                                       |
|      80 |       | boot    |         |                                       |
|      81 |       | boot    |         |                                       |
|      82 |       | boot    |         |                                       |
|      83 |       | boot    |         |                                       |
|      84 |       | boot    |         |                                       |
|      85 |       | boot    |         |                                       |
|      86 |       | boot    |         |                                       |
|      87 |       | boot    |         |                                       |
|      88 |       | boot    |         |                                       |
|      89 |       | boot    |         |                                       |
|      90 |       | boot    |         |                                       |
|      91 |       | boot    |         |                                       |
|      92 |       | boot    |         |                                       |
|      93 |       | boot    |         |                                       |
|      94 |       | boot    |         |                                       |
|      95 |       | boot    |         |                                       |
|      96 |       | boot    |         |                                       |
|      97 |       | boot    |         |                                       |
|      98 |       | boot    |         |                                       |
|      99 |       | boot    |         |                                       |
|     100 |       | boot    |         |                                       |
|     101 |       | boot    |         |                                       |
|     102 |       | boot    |         |                                       |
|     103 |       | boot    |         |                                       |
|     104 |       | boot    |         |                                       |
|     105 |       | boot    |         |                                       |
|     106 |       | no boot |         |                                       |
|     107 |       | boot    |         |                                       |
|     108 |       | boot    |         |                                       |
|     109 |       | boot    |         |                                       |
|     110 |       | boot    |         |                                       |
|     111 |       | boot    |         |                                       |
|     112 |       | boot    |         |                                       |
|     113 |       | boot    |         |                                       |
|     114 |       | boot    |         |                                       |
|     115 |       | boot    |         |                                       |
|     116 |       | boot    |         |                                       |
|     117 |       | boot    |         |                                       |
|     118 |       | boot    |         |                                       |
|     119 |       | boot    |         |                                       |
|     120 |       | boot    |         |                                       |

General DUT#3 behaviour: slower boot, pause after power on, and visible delay while 
lighting USRLED1-3 until SD-card boots. Might be caused by a different uboot edition
than DUT#1 and DUT#2.

** Test 2 results (DUT#1 pulldown removed)

| Boot no. | DUT#1 |
|        1 | boot  |
|        2 | boot  |
|        3 | boot  |
|        4 | boot  |
|        5 | boot  |
|        6 | boot  |
|        7 | boot  |
|        8 | boot  |
|        9 | boot  |
|       10 | boot  |
|       11 | boot  |
|       12 | boot  |
|       13 | boot  |
|       14 | boot  |
|       15 | boot  |
|       16 | boot  |
|       17 | boot  |
|       18 | boot  |
|       19 | boot  |
|       20 | boot  |
|       21 | boot  |
|       22 | boot  |
|       23 | boot  |
|       24 | boot  |
|       25 | boot  |
|       26 | boot  |
|       27 | boot  |
|       28 | boot  |
|       29 | boot  |
|       30 | boot  |
|       31 | boot  |
|       32 | boot  |
|       33 | boot  |
|       34 | boot  |
|       35 | boot  |
|       36 | boot  |
|       37 | boot  |
|       38 | boot  |
|       39 | boot  |
|       40 | boot  |
»  Substance:WordPress   »  Style:Ahren Ahimsa
© 2016 Mikini Services