Enlightenment Foundation Libraries on SBC-SAMA5D36
Products for consumer market, industrial control systems and other embedded devices often require a touch screen display to offer both, visual feedback to the user, as well as an input method to interact with the system. In order to achieve a nicely designed and responsive user interface, developers can write their own graphical library providing widgets like drop-down menus, list-views, check-boxes or simple buttons. However, this can be a very tedious and error-prone task. It is best to use one of the available graphic libraries like Qt, Cairo, EFL or also emWin, not to forget to mention nCurses for console-style block-oriented graphics.
Now which library should one select, especially in terms of limited hardware resources on embedded systems but also with a look forward to the look&feel as well as the usability on a touch screen? Besides, hardware restrictions, like the used CPU and its computing power, available RAM-size, free space in ROM, availability of a GPU with 2D/3D hardware accelerated graphics and the used operating system, should be taken into consideration. emWin for example, runs on bare-metal systems either with or without a RTOS whereas Qt or EFL need at least a running Linux (but not said that it is impossible to run them on bare-metal). Other decision affecting parameters are the availability of the library as open- or closed-source, as well as the costs for non-commercial and commercial products. In the following article we will take a brief look at the Enlightenment Foundation Libraries (EFL) of a small Cortex-A5 based single board computer from emtrion running a Debian Jessie based Linux: the SBC-SAMA5D36 .
Before getting into EFL itself, let’s dive a little bit into the actual kernel graphics subsystem and take a look at how Linux handles graphics and graphic-cards and which modules are involved. We will also look at the required user space drivers.
The Direct Rendering Infrastructure (DRI) handles graphics related data in kernel space as well as in user space. Kernel space includes the modules' Direct Rendering Manager (DRM), Kernel Mode Settings (KMS) and Graphics Execution Manager (GEM) whereas libDRM is the user space library .
The DRM is a hardware-specific driver with direct access to graphic cards. It handles DMA, resource locking, secure hardware access and some other aspects, for example multiple and simultaneous access to the graphics card and security . The DRM itself is divided into a generic DRM core, which handles hardware independent data and the specific DRM driver for a given graphics card. Both offer a set of ioctl functions allowing communication between hardware and the user psace part of the driver. KMS, on the other hand, handles mode setting calls to change, display resolution or color depths, for example. GEM is a memory management system for the graphic card memory.
On user space side is the libDRM driver. It is mostly a wrapper for the ioctl calls to the kernel DRM module and may contain helper/convenience functions to give developers an easier way to handle the graphics system without dealing with ioctl calls.
Display Servers and UI-Toolkit
The next tier in the Linux graphics system is represented by display server protocols like X-Server or Wayland and their corresponding implementations like X.Org and Xfree86 for X-Servers and weston and mutter for Wayland. Some of them use libDRM directly by implementing an own 2D rendering driver like Device Dependent X (DDX) in X.Org, others use 3D acceleration through the Mesa 3D libraries with opengl, also to perform 2D rendering. In addition, a compositing window manager is needed to do the compositing of different application windows on a desktop. Where X.Org needs a separate composite manager, display servers based on wayland have the compositing part integrated and are therefore called Wayland compositors. We aren't getting into detail at this point as it would go far beyond the scope of this article.
On top of the display servers and windows compositors, user interface toolkits are used. They provide a set of widgets like buttons, text boxes, labels, radio buttons and different layouts to automatically arrange the widgets, even when resizing the window. They also often provide some sort of event generation and handling. Most toolkits are available on different platforms like Windows, Linux and MacOS, intending to give the same look and feel on every platform. To write the application only once and use it on every supported platform is another advantage of cross-platform toolkits. EFL is one of the ui toolkits besides of the well known Qt and others.
The Enlightenment Foundation Libraries (EFL) are part of the Enlightenment Desktop project, which was initiated by Carsten ‘Rasterman’ Haitzler. They are small and portable and can also be used on small devices without powerful hardware. Some projects using the EF Libraries are mentioned on the enlightenment webpage . They list amongst others the Yellow Dog Linux for PS3, the Openmoko Freerunner smartphone and some Samsung gear. Samsungs Tizen , and therefore all products that are based on Tizen, also use EFL. It is completely developed using the C programming language and runs on frame buffer devices, Wayland, X11, Windows, OSX and others.
Applications based on EFL work like the IPO model, which stands for Input-Processing-Output. Simplified, each application has its own EFL Mainloop, where it waits for inputs like events, processes them and outputs the results to the GUI or other components.
EFL breaks down into a whole bunch of libraries, each of them dedicated to a specialized set of functions. Here we cover a small subset of libraries, only. More information can be taken from the EFL documentation .
|Ecore:||Ecore is the heart of the EFL. It provides the main loop and convenience functions for event handling, timers and other things.|
|Evas:||Evas is the canvas library and provides a canvas for the developer to draw his GUI. It doesn’t include any widgets. Display parameters like size and orientation are abstracted and there ist no need for the developer to know anything about it. Evas provides some basic object types like lines, rectangles and text objects and delivers a set of functions, like move and resize, to manipulate them.|
|Eina:||Eina is the base of all data types that EFL uses and implements, for example, lists, hash tables and string buffers. Besides that, it also contains some helpers or tools for runtime type checking, converters and others.|
|Eet:||Eet is known as a data handling library and allows writing data to a file that can optionally be compressed. It also allows random access to these files, even if they are compressed. Another nice feature is the serialization part of Eet, which allows serialization and parsing of self defined data types.|
|Edje:||Edje is responsible for the application layout and theme. The layout and reactions to events like mouse over, are described in an extra file that has to be compiled with the edje_cc compiler. Changing the look and feel of an application without recompiling it, is one advantage of this approach to split both theme and layout from the application.|
Elementary  is another library that should be mentioned. It is not part of the EFL itself but makes heavy use of it. Elementary provides a set of widgets, containers and some additional infrastructure modules. The infrastructure modules cover the (customizable) mouse cursor, fonts, gesture layer, widget resizing and other useful things. For a clean and scalable layout, Elementary offers containers of various types to arrange widgets in a certain way. There are layouts available like grids, panes, tables or simple boxes. And finally there are the widgets. Almost all common widgets are present, from simple ones like buttons, check boxes, progress bars and labels to complex ones like lists, a file selector, toolbars and a calendar. To get more information or view pictures of the widgets, take a look at the elementary webpage .
Both projects are regularly checked by the free OpenSource Coverity Scan . Coverity is a static tool checker that doesn’t need to run the code. It searches for null pointer dereferencing, memory leaks, deadlocks, best practice coding errors and much more. EFL and Elementary have both a very low defect density, which measures the number of defects per 1,000 lines of code. On the day writing this article (29.06.2015), EFL had a defect density of 0.11 and Elementary a defect density of 0.05. The qt-project on the other hand had a density of 0.25. A defect density of zero doesn’t mean that the software has no bugs, but it is a good quality indication of the code.
EFL on SBC-SAMA5D36
As embedded system we use the SBC-SAMA5D36 single board computer from emtrion. It’s populated with an ARM Cortex-A5 based SoC from Atmel, the SAMA5D36 , running at 536 MHz and has a total of 256 MB RAM and 256 MB NAND-Flash. A WVGA (800 x 480 pixel) sized LCD with capacitive touch screen is used. The root file system is based on Debian Jessie, which makes it very easy to install new software and to distribute built packages. For testing purposes, keyboard and mouse are connected to the two available USB-slots.
On SBC-SAMA5D36, hereinafter referred to as SBC, there are some restrictions regarding computation speed and graphics acceleration functions. The graphics driver is DRM-based and exports a simple frame buffer device. The integrated LCD controller contains some 2D acceleration features like a hardware cursor, overlays and image scaling, but they are only accessible through the DRM driver and not with the frame buffer device. That means, that we have two possible engines at the moment. The frame buffer without any hardware acceleration on the one side and the DRM at the other side. Starting an EFL based application with DRM as graphics engine on the serial terminal /dev/ttyS0 results in some errors regarding DRM and the app won't run. One way is to start the application on a virtual terminal (VT). That can either be done by using a USB keyboard connected to the sama and the terminal shown on the LCD or with openvt. Openvt opens a virtual terminal and runs the application on that terminal. So if we want a basic hardware acceleration, we need to run the application on a virtual terminal, otherwise the CPU has to do all calculations. NEON support is also missing as well as a dedicated video hardware decoder. That may lead to some performance issues, but we will see later how the system performs.
First of all, we need to build the EFL from scratch. There are some Debian packages available, but the version of EFL covered by those packages is 1.8.6 and the latest release of EFL has the version number 1.14.1. Regarding to the Enlightenment webpage, they should be built in a specific order (it is obvious to build Elementary after EFL as it depends on EFL). A complete description on how to build EFL and Enlightenment is available on the official site . The sources of EFL, Elementary and the applications were cloned from the official Enlightenment git repository .
- evas_generic_loaders (provides handling for filetypes like svg and pdf)
They mention also a bunch of dependencies to libraries and console programs like make, libdbus, openssl and others. They have to be installed first. After that, we can clone the official git repositories and checkout the latest stable branch v1.14.1. In addition we have to set some exports.
$ export PATH=/usr/local/bin:"$PATH"
$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:"$PKG_CONFIG_PATH"
$ export LD_LIBRARY_PATH=/usr/local/lib:"$LD_LIBRARY_PATH"
EFL has a lot of extras and can be build for many different display servers. We want to have a build for the frame buffer and disable all of the other possibilities like X. The sound can be disabled as the SBC has no sound output. Gstreamer and plug-ins need a lot of disk space which may not fit into the NAND flash in every case, so we disable it too. The SAMA5D3 lacks of NEON support, so it must also be disabled, otherwise we would get a “illegal instruction” program abort. The command line to build EFL looks as follows:
$ ./autogen.sh --disable-neon --enable-drm --enable-fb --with-opengl=none --without-x --with-x11=none --disable-tslib --disable-audio --disable-pulseaudio --disable-xim --disable-scim --disable-ibus --disable-gstreamer1 --disable-doc --disable-avahi --disable-xinput2 --with-crypto=gnutls --with-libgcrypt-prefix=arm-unknown-linux-gnueabihf --enable-i-really-know-what-i-am-doing-and-that-this-will-probably-break-things-and-i-will-fix-them-myself-and-send-patches-aba
If the EFL aren’t build with default settings, it bothers the developer with warnings, that this configuration isn’t tested and no one should write any bug reports to the EFL maintainer because of the modified settings. That’s why we need the last option in the configure command “--enable-i-really-know-what-i-am-doing-and-that-this-will-probably-break-things-and-i-will-fix-them-myself-and-send-patches-aba”.
After $make and $make install the EFL libraries are installed on the system. Now we can build the widget library Elementary. Here are no special options necessary. After $make and $make install Elementary is also ready to use.
To test the setup, we can use and build some of the example applications that are provided by the enlightenment developers, for example the ePhoto application or the Terminology terminal emulator. Beforehand we should export some settings Elementary can use. We want to use frame buffer and activate the mobile profile, which resizes buttons and other touchable parts of the UI and make it ready to use with touch screens.
$ export ELM_ENGINE=fb (or drm, if running on a VT)
$ export ELM_PROFILE=mobile
The last step is to activate dbus manually, as there is no X11 running and the dbus autolaunch relies on X11. There are two parts of EFL that use dbus for communication, Efreet and Ethumb.
$ eval `dbus-launch --auto-syntax`
During the tests we encountered problems with the touch interface. After taking a closer look at the implementation of the frame buffer related part and separating the input handling, there was a small part of code missing. The touch button event, with the event code 330, wasn’t used in the actual implementation. After adding this special event code and remapping it as a mouse button event with code 272, the touch events were recognized. The next problem was the position of the mouse cursor. The touch device is registered as an absolute position system. The corresponding sourcecode, evaluating the positions, got through the external touch event but never generated an internal event with the correct position for the application. After fixing this issue and generating the internal events, also the touch and mouse cursor position were correctly used and displayed. This fix was a quick work around only and does not provide multi-touch.
To test the performance of the SBC we use the Expedite benchmark suite, which is also available through the Enlightenment git repository. It includes icon scrolling, image smoothing, blending, cross-fading and displays text. To run the test, we need to enter the engine type as well as the resolution of the screen as arguments and tell it to run in full-screen mode.
$ expedite -e=fb -p=wvga -f
The benchmark suite measures Frames per Second (FPS) for each single benchmark and reports it to the command line. As there are over 100 separate benchmarks, the following results show only a few of them. The main developer of Enlightenment and EFL, Carsten Haitzler, also benchmarked some embedded systems in comparison to a desktop system . Table 1 shows the results of some benchmarks from the Expedite benchmark suite. The benchmarks were run on the SBC-SAMA5D36@536 MHz with 256 MB RAM and the frame buffer engine. We also used two different displays, one with QVGA (320x240 pixel) resolution and the other one with WVGA (800x480 pixel).
|Widget File Icons||18.81||5.66|
|Image Blend Unscaled||6.98||5.79|
|Image Blend Smooth Solid Scaled||23.43||13.84|
|Image Map Rotate||2.76||2.43|
|Image Map 3D 1||39.16||12.71|
|Image Quality Scale||49.86||13.11|
|Font Effect Blur (Alpha)||13.06||9.90|
During the test using the benchmark suite, the CPU was utilized at around 100 %. Running Ephoto uses the CPU only, if there runs a slide show or if the user interacts with the system. Otherwise the application utilizes the CPU with almost around 0 %. Interestingly, running htop besides the application to measure the CPU usage has some impact to the results of the benchmark, which then delivers around 5 % less frames per second than without running htop.
To measure the memory footprint of an application, smem  was used. It reports the memory usage as Unique Set Size (USS), Resident Set Size (RSS) and Proportional Set Size (PSS). USS is the unshared or private memory of an application, whereas RSS and PSS also include shared memory. The reported size of RSS may be confusing, because it accumulates the memory usage of all used shared libraries, even if other processes use those libs too. The better data delivers PSS, which divides the size for the shared libraries memory usage with the number of processes referring to that library. All values in Table 2 represent a snap-shot of the application at a given time and may vary. Especially the memory values of Expedite vary from benchmark to benchmark, mostly depending on the used widget and which images the application uses. With Ephoto a total of 20 JPEG-images with a resolution of 800x480 pixels and a size between 17 kB and 261 kB were used.
|Memory usage / KByte|
|Expedite||sliding through circular menu||~33 %||6520||6810||8348|
|Expedite||benchmarks running||~ 96 %||9104||9409||10904|
|Expedite||building thumbnails||~85 - 90 %||13240||15330||20616|
|Expedite||running slideshow||~20 - 50 %||21168||22056||24240|
|Expedite||execute ls multiple times||~60 %||26472||26755||28436|
Ephoto, an image viewer, and Terminology, a terminal emulator, were used for real world tests . The results of those tests are completely subjective. Terminology behaves very well. Calling the configuration menu with a right mouse click stops the system for some time, so it does for every setting in the settings menu, as the system has to gather some information. For example, setting the used font requires the SBC to collect all available fonts and generate a preview of them, which takes a few seconds. Same goes for the color picker. Showing the theme chooser also utilizes the CPU with around 95%, but there are many different animations.
Same counts for the Ephoto application. It needs around three seconds for startup. Generating thumbnails for all images within a folder takes quite some time while utilizing the CPU with around 85 to 90 %. Viewing the full image and switching between them is very smooth, except a small latency of maybe half a second until the image changes. The smoothness of the slideshow depends on the transition setting. Using a square transition, which starts with a small image in the middle of the screen and resizes it until it is as big as the screen, someone can see some missing frames whereas horizontal scrolling seems to behave better.
Conclusion and Outlook
The Enlightenment Foundation Libraries are a good alternative to Qt, although the Qt infrastructure with it’s QtCreator has some advantages. It runs out of the box on a SBC, for example. The performance isn’t that bad at all, as long as no fancy graphical stuff has to be calculated. So if you want to develop an application based on EFL, use a mostly static GUI. Avoid rapid changes of huge parts of the screen to avoid flicker and slow response, especially if you have to recalculate data in the background, like picture previews of before hidden images, font effects or other computational intensive things. Smaller graphical effects shouldn’t be a problem. That applies not only for EFL but also for other graphical libraries like Qt, as the SBC provides only reduced computing power compared with other systems like an iMX6.
As we thought, the SBC-SAMA5D36 is not usable for video playback and multimedia, as the processor lacks of every kind of accelerator like NEON or a dedicated video acceleration core. A test with Rage, the video playback application from Enlightenment, and some H.264 and MPEG2 encoded video files showed a nice slideshow with around three to ten seconds per picture. And thats not what we want.
To get a real comparison of EFL to other graphic libraries, an application would have to be written for EFL an Qt, in order to have a base for comparing frames per second, memory footprint and CPU usage on booth. The memory footprint of the applications used in this article give an overview of the EFL memory usage, but this data is only worth something, if it can be compared to another graphical library system. Tomasz Olszak did already tests by porting Elemines, a Minesweeper clone, to Qt . He measured the memory footprint on different systems as well as the startup time of both applications. His conclusion is, that EFL is slightly faster on startup and uses less memory on a 32 bit system, which was a difference of over 30 % for his application.
If you own a SBC-SAMA5D36 and want to test or use EFL with your own application, you can get the precompiled library as well as Elementary and the benchmark suite Expedite by installing it through apt-get. Just use the following command on your SBC.
$ apt-get install efl elementary expedite
The applications ephoto, equation and terminology are also avaliable through the emtrion Debian package repository. Install them with apt-get. Be aware, that the version numbers for the applications Ephoto, Equate, Expedite and terminology aren't their original ones, especially most of them don't have an explicit numbering scheme in the official git repository. So we used the version of EFL which was used to build the application against.
References and Links
- http://www.rasterman.com/index.php?page=News (News-entry from 2. January2009)