Tag : python

post image

How to make a snap package for lolcat with snapcraft on Ubuntu

In this post we are going to see how to make snap installation packages for a program called lolcat.

What you need for this tutorial? This tutorial is similar to Snap a python application. If you first follow that tutorial, then this one will reinforce what you have learned and expand to dealing with more programming languages than just Python.

What will you learn?

  1. How to deal with different source code repositories and get into the process of snapping them in a snap (pun intended).
  2. How to use the Snapcraft plugins for Python, golang, Rust and C/C++.
  3. How to deal with confinement decisions and when to select strict confinement or classic confinement.
  4. How to test the quality of an app before releasing to the Ubuntu Store
  5. What is this truecolor terminal emulator thing, what is lolcat and why it is cool (at least to a select few).

True-color terminal emulators

Terminal emulators like GNOME Terminal support the facility to display text in different colors.

You know this already because by default in Ubuntu you can see filenames in different colors, depending on whether they are executable (green), a directory (blue), a symbolic link (cyan) and so on. If you go 20+ years in the past, even the Linux console supported since then 256 colors.

What changed recently in terms of colors in the terminal emulators, is that newer terminal emulators support 16 million colors; what is described as true-color.

Here is an AWK script (of all scripting languages!) that shows a smooth gradient, from red to green to blue. In old terminal emulators and the Linux console, they were escape sequences to specify the colors, and you had to specify them by name. With the recent improvement among terminal emulators, it is now possible to specify the color by RGB value, thus 256*256*256 ~= 16 million different colors. You can read more about this in the article True Colour (16 million colours) support in various terminal applications and terminals. I found that AWK script there in this article. Try it one your Ubuntu as well!

Now, there is this Unix command called cat, which is used to print the contents of a text file in the terminal. cat /etc/passwd would show the contents of /etc/passwd. So, some people wrote an improved cat, called lolcat, that shows the content of files in a rainbow of colors.

In the screenshot above, the utility lolcat (a version we made and call lolcat-go) is used as a filter to a command (snapcraft –help), and colorizes the text that it receives with a nice rainbow of colors.

In practice, the lolcat utility merely sets the color for each character being printed. It starts with a random color and circles around the rainbow, heading diagonally towards the bottom-right. In the screenshot above, you can see the text snapcraft and Usage: in strong blue, and then diagonally (heading to bottom-right), the colors shift to green and then to orange.

Select which repositories to snap

The first implementation of the lolcat rainbow utility was probably by busyloop, written in Ruby. Since then, several others re-implemented lolcat into more programming languages. In this post we are going to snap:

  1. The Python lolcat by tehmaze.
  2. The golang lolcat by cezarsa.
  3. The Rust lolcat by ur0 (a version that uses Rust’s concurrency feature!)
  4. The C lolcat by jaseg (a version optimized for speed!)

Here we can see a) the URL of the source code, the available b) branches and c) tags. This is useful in the next section when we instruct Snapcraft which version of the source code to use.

In terms of versions among the four implementations, the Python lolcat has a recent tag 0.44, therefore we are using this tag and specify a version 0.44. For the rest, we are using the latest (master) checkout of their repositories, so we use the current date as a version number, in the form YYYYMMDD (for example, 20170226).

Completing the snapcraft.yaml metadata

When creating a snap, we need to write up a nice snap summary (less than 80 characters) and a description (less than 100 words). The summary will look like lolcat-python utility written in Python, and the description lolcat is a utility similar to the Unix “cat” command. lolcat adds rainbow colors to the text output.

In addition, we need suitable names for the four snaps. We are going to use lolcat-python, lolcat-go, lolcat-rust, and lolcat-c for each one of the four snaps.

All in all, here is the information we collected so far:

  1. Python: name is “lolcat-python“, version is “0.44“, summary: lolcat-python utility written in Python, description: lolcat-python is a utility similar to the Unix “cat” command. lolcat-python adds rainbow colors to the text output.
  2. Go: name is “lolcat-go“, version is “20170226“, summary: lolcat-go utility written in Go, description: lolcat-go is a utility similar to the Unix “cat” command. lolcat-go adds rainbow colors to the text output.
  3. Rush: name is “lolcat-rust“, version is “20170226“, summary: lolcat-rust utility written in Rust, description: lolcat-rust is a utility similar to the Unix “cat” command. lolcat-rust adds rainbow colors to the text output.
  4. C: name is “lolcat-c“, version is “20170226“, summary: lolcat-c utility written in C, description: lolcat-c is a utility similar to the Unix “cat” command. lolcat-c adds rainbow colors to the text output.

The final part in the metadata is the grade (whether it is devel or stable) and the confinement (whether it is devmode, classic or strict).

We select stable as the grade because we aim to add these snaps to the Ubuntu Store. If the grade is specified as devel, then the corresponding snap cannot be added to the stable channel (publicly available to all) of the Ubuntu Store. There is a section below on testing the four lolcat implementations and we will set accordingly the grade to either stable or devel, depending on the outcome of these tests.

We select initially devmode (DEVeloper MODE) as the confinement in order not to confine initially out snap. If our snap fails to run, we want to be sure it is some issue with our settings and not a byproduct of the choice of a stricter confinement. Once the snap can be built and run successfully, we change the confinement to either strict or classic and deal at that point with any issues that appear from there on.

Here is how the metadata look in the snapcraft.yaml configuration file for the Python variant of the snap.

Up to this point, all four snapcraft.yaml files, with metadata filled in, can be found at https://github.com/simos/lolcat-snap/tree/v1.0

Working on the “parts” section in snapcraft.yaml

We completed the easy metadata section, now it is time to work on the parts: section of snapcraft.yaml. (After the parts: section work, we just need the apps: section, where we expose the generated executable to the users of the snap).

Initially, snapcraft init will generate a stock snapcraft.yaml file, which has a stub parts: section. Here it how it looks,

    # See 'snapcraft plugins'
    plugin: nil

It defines the start of the parts: section. Then, a name, my-part: (we choose this name) is defined. Finally, the contents of my-part: are listed, here in two lines. The first line of my-part: is a comment (starts with #) and the second specifies the plugin, which is nil (it is reserved, and does nothing).

Note how these lines are aligned vertically. First is parts: at the start of the line, then two columns further, is my-part:. Then, two more columns further, we get the comment and the plugin: directive, both on the same column. We use spaces and not tabs. If you get any errors later on, check that you follow properly the formatting. You can have the levels separated by more than two columns, if you wish. But make sure that the same level lines are aligned on the same column.

Let’s figure out the initial versions of the parts:. Let’s do Python first!

    source: https://github.com/tehmaze/lolcat.git
    source-tag: '0.44'
    plugin: python

We used the name lolcat-python: (our choice) for the name of this parts:. In there, we specify the source: for the source code repository, and any branches or tags that may be relevant. As we saw above, we work on the 0.44 tag. Finally, the source is written in Python, and we select the python plugin. (We will figure out later if we need to specify which of Python 2 or Python 3, if we get a relevant error).

Here is the Go,

    source: https://github.com/cezarsa/glolcat.git
    plugin: go

Fairly expected. We do not specify a version, therefore it will use the latest snapshot of the source at the time of running snapcraft. We chose the name locat-go, and the golang plugin is called go.

Time for Rust,

    source: https://github.com/ur0/lolcat.git
    plugin: rust

Again, very similar to the above. We do not specify a specific source code version (there isn’t any tag or stable branch in the repository). We chose the name lolcat-rust, and the Rust plugin in snapcraft is called rust.

Finally, the version written in C,

    source: https://github.com/jaseg/lolcat.git
    plugin: make

Again, we chose a name for this part, and it’s lolcat-c. Then, specified the URL for the source code. As for the plugin, we wrote make, although the source is written in C. Actually, the plugin: directive specifies how to build the source code. The make plugin does the make; make install sequence for us, and it’s our first choice when we see an existing Makefile file in a repository.

At this stage, we have four initial versions of configuration files. We are ready to try them out and deal with any errors that may appear.

Working on the Python version

Here is the configuration (snapcraft.yaml) for the Python variant up to this point,

name: lolcat-python
version: '0.44'
summary: lolcat utility written in Python
description: |
  lolcat-python is a utility similar to the Unix "cat" command. 
  lolcat-python adds rainbow colors to the text output.
  The source code is available at https://github.com/tehmaze/lolcat

grade: stable
confinement: devmode

    source: https://github.com/tehmaze/lolcat.git
    source-tag: '0.44'
    plugin: python

Let’s run snapcraft prime. The prime parameter will perform the necessary compilation and will put the result in the prime/ subdirectory. Since snapcraft prime does only the initial compilation steps, it is more convenient to use for repeated tests. Once it works with snapcraft prime, we would be ready in a single extra step to produce the final .snap file (by running snapcraft, no parameters).

$ snapcraft prime
Preparing to pull lolcat-python 
Pulling lolcat-python 
Installing collected packages: pip, six, pyparsing, packaging, appdirs, setuptools, wheel
Successfully downloaded lolcat
Preparing to build lolcat-python 
Building lolcat-python 
Successfully built lolcat
Successfully installed lolcat-0.44
Staging lolcat-python 
Priming lolcat-python 
$ _

It worked beautifully! Any result is in the prime/ subdirectory, and we can identify even the executable.

$ ls prime/
bin/  etc/  lib/  meta/  snap/  usr/
$ ls prime/bin
$ _

We are now ready to create the apps: section of the snapcraft.yaml configuration file, in order to expose the executable to the users,

    command: lolcat

The name: of this snap is lolcat-python, and the name we chose in the apps: section is the same, lolcat-python. Since they are the same, once we install the snap, the newly available command will be called lolcat-python. If there was a difference between the two (for example, name: lolcat-python and apps: lolcat: […], then the command for the users would end up being something like lolcat-python.lolcat, which may not be desirable.

Working on the Go version

Here is the configuration (snapcraft.yaml) for the Go variant up to this point,

name: lolcat-go
version: '20170226'
summary: lolcat utility written in golang
description: |
  lolcat-go is a utility similar to the Unix "cat" command.
  lolcat-go adds rainbow colors to the text output.
  The source code is available at https://github.com/cezarsa/glolcat

grade: devel
confinement: devmode

    source: https://github.com/cezarsa/glolcat.git
    plugin: go

Let’s run snapcraft prime and see how it goes.

$ snapcraft prime
Preparing to pull lolcat-go 
Building lolcat-go 
[...]Staging lolcat-go 
Priming lolcat-go 
$ ls prime/bin/
$ _

Lovely! The go plugin managed to make sense of the repository and compile the Go version of lolcat. The generated executable is called glolcat.git. Therefore, the apps: section looks like

    command: glolcat.git

In this way, once we install the Go snap of lolcat, there will be an executable lolcat-go exposed to the users. This executable will be the glolcat.git that was generated by the Snapcraft go plugin in prime/bin/. Snapcraft tries to find the final executable in either prime/ or prime/bin/, therefore we do not need to specify specifically bin/glolcat.git.

Working on the Rust version

Here is the configuration (snapcraft.yaml) for the Rust variant up to this point,

name: lolcat-rust
version: '20170226'
summary: lolcat utility written in Rust
description: |
  lolcat-rust is a utility similar to the Unix "cat" command.
  lolcat-rust adds rainbow colors to the text output.
  The source code is available at https://github.com/ur0/lolcat

grade: stable
confinement: devmode

    source: https://github.com/ur0/lolcat.git
    plugin: rust

Let’s run snapcraft prime and see how it goes.

$ snapcraft prime
Preparing to pull lolcat-rust 
Downloading 'rustup.sh'[================] 100%
Preparing to build lolcat-rust 
Building lolcat-rust 
Staging lolcat-rust 
Priming lolcat-rust 
$ ls prime/bin/
$ _

Without a hitch! The rust plugin noticed that we do not have the Rust compiler, and downloaded it for us! Then, it completed the compilation and the final executable is called lolcat. Therefore, the apps: section looks like

    command: lolcat

The apps: section will expose the generated lolcat executable by the name lolcat-rust. Since the name: field is also lolcat-rust, the resulting executable once we install the snap will be lolcat-rust as well. Again, Snapcraft looks by default into both prime/ and prime/bin/ for the command:, therefore it is not required to write specifically command: bin/lolcat.

Working on the C (make) version

Here is the configuration (snapcraft.yaml) for the C (using Makefile) variant up to this point,

name: lolcat-c
version: '20170226'
summary: lolcat utility written in C
description: |
  lolcat-c is a utility similar to the Unix "cat" command.
  lolcat-c adds rainbow colors to the text output.
  The source code is available at https://github.com/jaseg/lolcat

grade: stable
confinement: devmode

    source: https://github.com/jaseg/lolcat.git
    plugin: make

Let’s run snapcraft prime and see how it goes.

$ snapcraft prime
Preparing to pull lolcat-c 
Pulling lolcat-c 
Submodule 'memorymapping' (https://github.com/NimbusKit/memorymapping) registered for path 'memorymapping'
Submodule 'musl' (git://git.musl-libc.org/musl) registered for path 'musl'
Cloning into 'memorymapping'...
Cloning into 'musl'...
Building lolcat-c 
make -j4
"Using musl at musl"
cd musl; ./configure
[...]sh tools/musl-gcc.specs.sh "/usr/local/musl/include" "/usr/local/musl/lib" "/lib/ld-musl-x86_64.so.1" > lib/musl-gcc.specs
printf '#!/bin/sh\nexec "${REALGCC:-gcc}" "$@" -specs "%s/musl-gcc.specs"\n' "/usr/local/musl/lib" > tools/musl-gcc
chmod +x tools/musl-gcc
make[1]: Leaving directory '/home/myusername/SNAPCRAFT/lolcat-snap/c/parts/lolcat-c/build/musl'
Command '['/bin/sh', '/tmp/tmpetv6irll', 'make', '-j4']' returned non-zero exit status 2
Exit 1
$ _

Well, well. Finally an error!

So, the C version of lolcat was written using the musl libc library. This library was referenced as a git submodule, and snapcraft followed the instructions of the Makefile to pull and compile musl!

Then, for some reason, snapcraft fails to compile lolcat using a musl-gcc wrapper for this newly compiled musl libc. As if we need to specify that the make ; make install sequence should instead be make lolcat; make install. Indeed, looking at the instructions at https://github.com/jaseg/lolcat, this appears to be the case.

Therefore, how do we tell snapcraft that for the make plugin, it should run make lolcat instead of the standard make?

By reading the documentation of the Snapcraft make plugin, we find that

- make-parameters:
 (list of strings)
 Pass the given parameters to the make command.

Therefore, the parts: should look like

    source: https://github.com/jaseg/lolcat.git
    plugin: make
    make-parameters: [lolcat]

We enclose lolcat in brackets ([lolcat]) because make-parameters accepts a list of strings. If it accepted just a string, we would not need those brackets.

Let’s snap prime again!

$ snapcraft prime
Preparing to pull lolcat-c 
Building lolcat-c 
make lolcat -j4
"Using musl at musl"
chmod +x tools/musl-gcc
gcc -c -std=c11 -Wall -g -Imusl/include -o lolcat.o lolcat.c
Staging lolcat-c 
Priming lolcat-c 
$ ls prime/bin
ls: cannot access 'prime/bin': No such file or directory
Exit 2
$ ls prime/
censor*  command-lolcat-c.wrapper*  lolcat*  meta/  snap/
$ _

Success! The lolcat executable has been generated, and in fact it is located in prime/ (not in prime/bin/ as in all three previous cases). Also, it took 1 minute and 40 seconds to produce the executable on my computer!

Hmm, there is also a censor executable. I wonder, what does it do?

$ ls -l | ./censor 
█▄█▄█ ██
█▄▄▄▄█▄▄█▄ █ ▄▄▄▄ ▄▄▄▄ █████ ███  ██ ██:██ ▄▄▄▄▄▄
█▄▄▄▄█▄▄█▄ █ ▄▄▄▄ ▄▄▄▄   ███ ███  ██ ██:██ ▄▄▄▄▄▄███▄█▄▄██▄.▄▄▄▄▄▄▄
█▄▄▄▄█▄▄█▄ █ ▄▄▄▄ ▄▄▄▄ █████ ███  ██ ██:██ █▄█▄▄█
█▄▄▄▄▄▄▄█▄ █ ▄▄▄▄ ▄▄▄▄  ████ ███  ██ ██:██ ▄▄█▄
█▄▄▄▄▄▄▄█▄ █ ▄▄▄▄ ▄▄▄▄  ████ ███  ██ ██:██ ▄▄▄▄
$ _

This censor executable is a filter, just like lolcat, and what it does is that it replaces any input letters with block characters. Like censoring the text. Let’s expose both lolcat and censor in this snap! Here is the apps: section,

    command: lolcat
    command: censor

For lolcat, since both the name: and the apps: command are named lolcat-c, the final executable will be lolcat-c. For censor, the executable will be named lolcat-c.censor (composed from the name: and the name we put in apps:).

Which confinement, strict or classic?

When we are about to publish a snap to the Ubuntu Store, we need to decide which confinement to apply, strict or classic.

The strict confinement will by default fully restrict the snap and it is up to us to specify what is allowed. For example, there will be no network connectivity, and we would need to explicitly specify it if we want to provide network connectivity.

The strict confinement can be tricky if the snap needs to read files in general. If the snap were to read file only from the user’s home directory, then that’s easy because we can specify plugs: [home] and be done with it.

In our case, lolcat can be used in two ways,

Note the different colors; the lolcat implementations start every time with a random initial rainbow color

In the first command, lolcat-go requires read access for the file /etc/lsb-release. This would be tricky to specify with the strict confinement, and we would need to use the classic confinement instead.

In the second command, lolcat-go is merely used as a filter. The executable is self-contained and does not read any files whatsoever. Because the cat command does the reading for lolcat.

Therefore, we can select here the strict confinement as long as we promise to use lolcat as a filter only. That is, lolcat would run fully confined! And it will fail if we ask it to read a file like in the first command.

Final configuration files

Here are the four final configuration files! I highlight below what really changes between them (bold+italics).

$ cat python/snap/snapcraft.yaml 
name: lolcat-python
version: '0.44'
summary: lolcat utility written in Python
description: |
  lolcat-python is a utility similar to the Unix "cat" command. 
  lolcat-python adds rainbow colors to the text output.
  The source code is available at https://github.com/tehmaze/lolcat

grade: stable
confinement: strict

    command: lolcat

    source: https://github.com/tehmaze/lolcat.git
    source-tag: '0.44'
    plugin: python

$ cat go/snap/snapcraft.yaml 
name: lolcat-go
version: '20170226'
summary: lolcat utility written in golang
description: |
  lolcat-go is a utility similar to the Unix "cat" command.
  lolcat-go adds rainbow colors to the text output.
  The source code is available at https://github.com/cezarsa/glolcat

grade: devel
confinement: strict

    command: glolcat.git

    source: https://github.com/cezarsa/glolcat.git
    plugin: go

$ cat rust/snap/snapcraft.yaml 
name: lolcat-rust
version: '20170226'
summary: lolcat utility written in Rust
description: |
  lolcat-rust is a utility similar to the Unix "cat" command.
  lolcat-rust adds rainbow colors to the text output.
  The source code is available at https://github.com/ur0/lolcat

grade: stable
confinement: strict

    command: lolcat

    source: https://github.com/ur0/lolcat.git
    plugin: rust

$ cat c/snap/snapcraft.yaml 
name: lolcat-c
version: '20170226'
summary: lolcat utility written in C
description: |
  lolcat-c is a utility similar to the Unix "cat" command.
  lolcat-c adds rainbow colors to the text output.
  The source code is available at https://github.com/jaseg/lolcat

grade: stable
confinement: strict

    command: lolcat
    command: censor

    source: https://github.com/jaseg/lolcat.git
    plugin: make
    make-parameters: [lolcat]
$ _

Quality testing

We have four lolcat snaps available to upload to the Ubuntu Store. Let’s test them first to see which ones are actually good enough to make it today to the Ubuntu Store!

A potential problem with filters like lolcat, is that they may not know how to deal with some UTF-8-encoded strings. In the UTF-8 encoding, text in English like what you read now, is encoded just like ASCII. Problems may occur when you have text in other scripts, where each Unicode character would occupy more than one byte. For example, each character in “Γεια σου κόσμε” is encoded as two bytes in the UTF-8 encoding. Furthermore, there are characters that are stored in Plane 1 of Unicode (Emojis!) which require the maximum of four bytes to encode in UTF-8. Here is an example of emoji characters, “🐧🐨🐩🐷🐸🐹🐺🐼🏴👏” (if they appear as empty boxes, you do not have the proper fonts installed.

What we are seeing here, is that the current implementations of lolcat for Python and Go do not work with non-ASCII encodings. At the moment, they can only colorize text in English.

On the other hand, the Rust and C implementations work fine on all Unicode characters!

There is another issue to watch for. Specifically,

The Unix ls command takes care to understand when you are redirecting the output, and if you do so, by default it does not colorize the output. This is a very useful default for ls because in most cases you want clean text files with no escape sequences for the color. To further our testing, let’s see how well the remaining two lolcat implementations deal with content that happen to already have escape sequences (for example, for color),

We generated a text file (cat-color.txt) with color escape sequences. The Rust implementation choked on them, while the C implementation worked fine!

We have a winner! The C implementation of lolcat will be published in the Ubuntu Store, in the stable channel (available to all). The Rust implementation will be published in the candidate channel (pending a fix when dealing with text that already has escape sequences), while the Python and Go implementations will be published in the edge channels. (Note: if the authors of these lolcat implementations are reading this, my aim here is to demonstrate the different channels. I plan to write a follow-up post on how to fix these issues in the source code and how to re-release all snaps to the stable channel.)

Let’s publish to the Ubuntu Store

The documentation page Publish your snap from snapcraft.io explains the details on publishing a snap on the Ubuntu Store.

In summary, when releasing a snap to the Ubuntu Store, you specify in which channel(s) you want it to be in:

  1. stable channel, for publicly available snaps. These can be searched when running snap find, and will be installed when you run snap install.
  2. candidate channel, for snaps that are very soon to be released to the stable channel. For this and the following channels, you need to specify explicitly the channel when you try to install. Otherwise, you will get the error that the snap is not found.
  3. beta channel, for beta versions of snaps.
  4. edge channel, for testing versions of snaps. This is the furthest away from stable.

If you release a snap in the stable channel, it implies that it will be shown in those below. Or, if you release a snap in the beta channel, it will appear as well in the channel below (edge).

Furthermore, when we were filling in the metadata of the snapcraft.yaml configuration files, there was a field called grade, with options for either stable or devel. If the grade has been specified as devel, then the snap can only be installed in either the beta or edge channels.

So, this is what we are going to do:

  1. The C implementation, lolcat-c, will have grade: stable, and get published in the stable channel of the Ubuntu Store.
  2. The Rust implementation, lolcat-rust, will have grade: devel, and get published in the beta channel of the Ubuntu Store.
  3. The Go and Python implementations, lolcat-go and lolcat-python, will have the grade: devel and get published in the edge channel of the Ubuntu Store.

We have already ran snapcraft login, and logged in with our Ubuntu Single SignOn (SSO) account.

We successfully registered the names on the Ubuntu Store, and then we pushed the snap files. We note down the revision, which is Revision 1 for each one of them.

The snapcraft release command looks like

  snapcraft [options] release <snap-name> <revision> <channel>

The snapcraft release command instructs the Ubuntu Store to release the recently pushed .snap file into the specified channels.

How to install them

Here we install lolcat-c.

First we perform a search for lolcat. The snap lolcat-c is found. There is a feature in the snap find command, that if we were to search for lolcat-c, it would show results for both lolcat and c (shows irrelevant results).

We then run the snap info command to show the Ubuntu Store info for the snap lolcat-c.

Then, we install lolcat-c, by running snap install lolcat-c.

Finally, we run snap info again, and we can see more information since the snap has just been installed. In particular, we can see that there is the additional command lolcat-c.censor. Oh, and then snap is just 24kB in size.

Let’s install the Rust implementation!

The Rust implementation of lolcat can be found in the beta and edge channels. In order to install it, we need to specify the correct channel (here, either beta or edge). And that’s it!

The Go implementation of lolcat is in the edge channel. We install and test it successfully.

Finally, the Python implementation of lolcat, installed from the edge channel.

Hope you enjoyed the post!

post image

How to create a snap with snapcraft for howdoi (CLI utilility for stackoverflow)

In the tutorial How to create a snap for how2 (stackoverflow from the terminal) in Ubuntu 16.04 we saw how to create a snap with snapcraft for the CLI utility called how2. That was a software based on nodejs.

In this post we will repeat the process for another CLI utility called howdoi by Benjamin Gleitzman, which does a similar task with how2 but is implemented in Python and has a few usability differences as well. howdoi does not have yet a package in the repositories of Ubuntu, either.

Since we covered already the details in How to create a snap for how2 (stackoverflow from the terminal) in Ubuntu 16.04, this post would be more focused, and shorter. 🙂


Reading through https://github.com/gleitz/howdoi we see that howdoi

  1. is software based on Python (therefore: plugin: python)
  2. requires networking (therefore: plugs: [network])
  3. and has no need to save files (therefore it does not need access to the filesystem)

Crafting with snapcraft

Let’s start with snapcraft.

$ mkdir howdoi
$ cd howdoi/
$ snapcraft init
Created snap/snapcraft.yaml.
Edit the file to your liking or run `snapcraft` to get started

Now we edit snap/snapcraft.yaml and here are our changes (in bold) from the initial generated file.

$ cat snap/snapcraft.yaml 
name: howdoi # you probably want to 'snapcraft register <name>'
version: '20170207' # just for humans, typically '1.2+git' or '1.3.2'
summary: instant coding answers via the command line # 79 char long summary
description: |
  Are you a hack programmer? Do you find yourself constantly Googling 
  for how to do basic programing tasks?
  Suppose you want to know how to format a date in bash. Why open your browser 
  and read through blogs (risking major distraction) when you can simply 
  stay in the console and ask howdoi.

grade: stable # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

    command: howdoi
    plugs: [network]

    plugin: python
    source: https://github.com/gleitz/howdoi.git

First, we selected to use the name howdoi because again, it’s not a reserved name :-). Also, we registered it with snapcraft,

$ snapcraft register howdoi
Registering howdoi.
Congratulations! You're now the publisher for 'howdoi'.

Second, we did not notice a particular branch or tag for howdoi, therefore we put the date of the snap creation.

Third, the summary and the description are just pasted from the Readme.md of the howdoi repository.

Fourth, we select the grade stable and enforce the strict confinement.

The apps: howdoi: command: howdoi is the standard sequence to specify the command that will be exposed to the user. The user will be typing howdoi and the command howdoi inside the snap will be invoked.

The parts: howdoi: plugin: python source: … is the standard sequence to specify that the howdoi that was referenced just earlier, is software written in Python and the source comes from this github repository.

Let’s craft the snap.

$ snapcraft 
Preparing to pull howdoi 
Pulling howdoi 
Preparing to build howdoi 
Building howdoi 
Successfully built howdoi
Installing collected packages: howdoi, cssselect, Pygments, requests, lxml, pyquery, requests-cache
Successfully installed Pygments-2.2.0 cssselect-1.0.1 howdoi-1.1.9 lxml-3.7.2 pyquery-1.2.17 requests-2.13.0 requests-cache-0.4.13
Staging howdoi 
Priming howdoi 
Snapping 'howdoi' |                                                                       
Snapped howdoi_20170207_amd64.snap
$ snap install howdoi_20170207_amd64.snap --dangerous
howdoi 20170207 installed
$ howdoi format date bash
DATE=`date +%Y-%m-%d`
$ _

Beautiful! It worked!

Publish to the Ubuntu Store

Let’s publish the snap to the Ubuntu Store. We are going to push the file howdoi_20170207_amd64.snap and then check that it has passed the automatic checking. Once it has done so, we release to the stable channel.

$ snapcraft push howdoi_20170207_amd64.snap 
Pushing 'howdoi_20170207_amd64.snap' to the store.
Uploading howdoi_20170207_amd64.snap [=============================================================] 100%
Ready to release!|                                                                                       
Revision 1 of 'howdoi' created.

Just a reminder: We can release the snap to the stable channel simply by running snapcraft release howdoi 1 stable. The alternative to this command, is to do all the following through the Web.

We log in into https://myapps.developer.ubuntu.com/ to check whether snap is ready to publish. In the following screenshots, you would click where the arrows are showing. See the captions for explanations.

Here is the uploaded snap in our account page in the Ubuntu Store. The snap was uploaded using snapcraft, although it is also possible to uploaded from the account page as well.


The package (the snap) is ready to publish, because it passed the automated tests and was not flagged for manual review.

By default, the package has not been released to a channel. We click on Release in order to select which channels to release it to.

For this specific package, we select the stable channel. It is not necessary to select the other channels, because by default a higher channel implies those below. Then, click on the Release button.

The package got released, and it shown it got released in stable, candidate, beta and edge (we selected stable, but the rest are implied because “stable” beats the rest.) Note that the Package status has changed to “Published”, and we have the option to Unpublish or even Make private. Ignore the arrow, it was pasted by mistake.

Μαθήματα γλώσσας προγραμματισμού Python στα ελληνικά

Στο φόρουμ του Ubuntu-gr έχουν ξεκινήσει μαθήματα για τη γλώσσα προγραμματισμού Python.

Το πρώτο μάθημα είναι για τη ρύθμιση του περιβάλλοντος Python στο σύστημά σας, οπότε μπορείτε εύκολα να λάβετε μέρος και να παρακολουθήσετε τα επόμενα βήματα.

Μέσω του φόρουμ του Ubuntu-gr γίνεται και η μετάφραση στα ελληνικά ενός οδηγού εκμάθησης της γλώσσας Python, που θα χρησιμοποιηθεί ως ύλη για τα μαθήματα. Η μετάφραση έχει σχεδόν ολοκληρωθεί.

Δείτε τα μαθήματα Python του φόρουμ Ubuntu-gr.

Ο Κώστας Τσακάλογλου αναφέρθηκε στα μαθήματα αυτά πριν λίγες μέρες.

The Keyboard Layout Editor

Update Dec 2010: Get the latest version of the Keyboard Layout Editor from https://github.com/simos/keyboardlayouteditor

(this entry is a repost, the original was lost in a database mishap.)

As part of the 2008 GSoC program, I worked on a Keyboard Layout Editor for the X.Org Foundation.

The Keyboard Layout Editor (KLE) is an application that allows you to create keyboard layouts for the X.Org server, commonly found in the Linux, OpenSolaris, *BSD, etc Desktops.

My mentor was Sergey Udaltsov, maintainer of xkeyboard-config, the Keyboard Indicator applet in GNOME, supporting libraries for keyboard layouts and much more. I had great help and Sergey was very supportive. Highly recommended mentor for your GSoC’09 project.

The Keyboard Layout Editor showing a layout

The Keyboard Layout Editor showing a layout

The screenshot above shows the main window of the program; a keyboard with blank layout (keys are empty), a section Add to layout with items that can be used to populate the layout, and a section for the description of the layout (Layout details).

There are typically two workflows; first you start off with a blank layout and you add Unicode characters, dead keys, include files, then you save.

The other workflow is to start with an appropriate existing layout as a base, then add more characters, make changes, etc.

It might be strange to talk about different workflows, but in terms of usability it’s important provide assistance for such cases. For example, having tooltips is important when a person starts off with a new layout.

Using the Keyboard Layout Editor

Using the Keyboard Layout Editor

Here we started with a blank layout; we click on Start Character Map, then locate the characters you need, and drag and drop them to the appropriate keys. Each key is composed of four parts, and we number these from 1 to 4. The way we count is quite peculiar,

  1. bottom left, when you press the key as is (key)
  2. top left, when you press the key with Shift (Shift + key)
  3. bottom right, when you press the key with AltGr (AltGr + key)
  4. top right, when you press the key with Shilft+AltGr (Shift + AltGr + key)
Analysis of a key

Analysis of a key

This is my entry to the most engineered diagram competition.

The dead keys relate to diacritic marks such as grave and acute. Since they are too small to see, we present them next to a D letter (D for Dead key). In some cases I could not find a character equivalent to the diacritic mark, so I put ?, therefore it looks like D?. If you put the mouse pointer over the key, you can see the full details in the tooltip.

Including files

Including files

In many cases, there exist layouts/variants that contain most of the characters you want to add. In this case, you add and enable in the Include files section. You can then override any of those characters by dragging and dropping to the layout.

At this stage in the blog post, it is important to clarify the notions of a layout and a variant. The two are quite similar and the distinction is messy when trying to explain to the end-user. The French layout file is fr, which contains several variants (distinct groups of mappings of physical keys to Unicode characters). When you are actually talking about a French keyboard layout, you are actually referring to the default variant of the «fr» file. Oftentimes people refer to the «fr» file as a whole as the French layout. You can also pick a non-default variant of the layout file, and call it your layout.

The way I would like to define layout and variant is this: a layout refers to the default variant of the layout file. This is consistent to the fact that distributions pick the default variant in the settings so it’s what get the most visibility, or when users select a new layout, they are presented with the default setting first. Regarding layouts in general, it is important for different languages/scripts to make effort that the default layout is updated and includes extra useful and relevant characters.

The new Greek keyboard layout

The new Greek keyboard layout

This is the updated Greek keyboard layout, and is the near-final version that we are planning to submit to xkeyboard-config. It adds Greek Polytonic to the existing Greek layout.  It does not make changes to the previous default layout, so users will not be unpleasantly surprised. It also adds all sort of characters that are found in the Greek Unicode block.

In this post I simplified some of the terms/description. If I went a bit too far, please correct me and I’ll update in-place.

Update 8th Sep 08: What are the plans for further development of the layout editor;

  • Increase the user base and get more people trying out the editor. This requires some more cleanup of the code, more instructions on how to run it youselves, and get people to provide feedback. An open-source project without users is not a successful project.
  • Make it easier for developers to contribute on the project. If you use Eclipse, you can install pydev, antlr3ide, mylyn, subclipse, and you can do the full development from within the cozy Eclipse environment. These need documentation.
  • The Issues page at the project has about ten items. This list needs to be reduced.
  • The natural place for users of the layout editor is the http://listserv.bat.ru/xkb/List.html mailing list. We need to promote the editor there, and get examples of users actually using it to maintain layouts.
  • An issue that plagues some users is when they need compose sequences to generate characters that no pre-composed forms exist. If users really need this (mainly Latin and Cyrillic scripts, complex scripts), it can be adapted to the UI.
  • It is technically easy to adapt the editor so that it produces XML layouts. Considering the state of XKB-atkins, this may not be a top priority at the moment. libxml2 comes with the MIT license, so in license terms it would be OK. Not sure if it is OK to link libxml2 to the X.org server. It might actually solve the slow parsing of the configurations files and the issues of xkbcomp.
  • At the moment the default geometry is a somewhat generic keyboard. In addition, I deactivated several keys (such as the function keys), in order not to confuse users (you can activate with a small change in the code). The keyboard can be expanded to a full 105-keys style. A related project would be to figure out an efficient way to edit those geometry files, and make the keyboard customised. If people start creating layouts with the editor, they will certainly love to edit geometry files!

ANTLR grammar for XKB, and Relax NG schema (draft)

I completed the ANTLRv3 grammar for symbols/ configuration files of XKB. The grammar can parse and create the abstract syntax tree (AST) for all keyboard layouts in xkeyboard-config.

ANTLRv3 helps you create parsers for domain specific languages (DSL), an example of which is the configuration files in XKB.

Having the ANTLRv3 grammar for a configuration file allows to generate code in any of the supported target lagnuages (C, C++, Java, Python, C#, etc), so that you easily include a parser that reads those files. Essentially you avoid using custom parsers which can be difficult to maintain, or parsers that were generated with flex/bison.

On a similar note, here is the grammar to parse Compose files (such as en_US.UTF-8/Compose.pre). I am not going to be using in the project for now, but it was fun writing it. The Python target takes 18s to create the AST for the >5500 lines of the en_US.UTF-8 compose file, on a typical modern laptop.

I am also working on creating a RelaxNG schema for the XKB configuration files (those under symbols/). There is a draft available, which needs much more work.The Relax NG book by Eric van de Vlist is very useful here.

The immediate goal is to use the code generated by ANTLR to parse the XKB files and create XML files based on the Relax NG schema. I am using Python, and there are a few options; the libxml2 bindings for Python, and PyXML. The latter has more visible documentation, but I think that I should better be using the former.

Update: lxml appears to be the nice way to use libxml2 (instead of using directly libxml2).

Parsing XKB files with antlr

antlr (well, antlr3) is an amazing tool that replaces lex/flex, yacc/bison.

One would use antlr3 if they want to deal with Domain-Specific Languages (DSL), an example of which are the text configuration files.

In our case, we use antlr3 to parse some of the XKB configuration files, those found in /etc/X11/xkb/symbols/??.

Our aim is to be able to easily read and write those configuration files. Of course, once we have them read, we do all sorts of processing.

The stable version of antlr3 is 3.0.1, which happened to give lots of internal errors. It has not been very useful, so I tried a few times the latest beta version 3.1b, and eventually managed to get it to work. If I am not mistaken, 3.1 stable should be announced in a few days.

When using antlr, you have the choice of several target languages, such as Java, C, C++ and Python. I am using the Python target, and the latest version that is available from the antlr3 repository.

Here is the tree of the gb layout file,

tree = (SECTION (MAPTYPE (MAPOPTIONS partial default alphanumeric_keys xkb_symbols) (MAPNAME “basic”)) (MAPMATERIAL (TOKEN_INCLUDE “latin”) (TOKEN_NAME Group1 (VALUE “United Kingdom”)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 quotedbl twosuperior oneeighth)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling threesuperior sterling)) (TOKEN_KEY (KEYCODEX AE04) (KEYSYMS 4 dollar EuroSign onequarter)) (TOKEN_KEY (KEYCODEX AC11) (KEYSYMS apostrophe at dead_circumflex dead_caron)) (TOKEN_KEY (KEYCODEX TLDE) (KEYSYMS grave notsign bar bar)) (TOKEN_KEY (KEYCODEX BKSL) (KEYSYMS numbersign asciitilde dead_grave dead_breve)) (TOKEN_KEY (KEYCODEX LSGT) (KEYSYMS backslash bar bar brokenbar)) (TOKEN_INCLUDE “level3(ralt_switch_multikey)”))) (SECTION (MAPTYPE (MAPOPTIONS partial alphanumeric_keys xkb_symbols) (MAPNAME “intl”)) (MAPMATERIAL (TOKEN_INCLUDE “latin”) (TOKEN_NAME Group1 (VALUE “United Kingdom – International (with dead keys)”)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 dead_diaeresis twosuperior onehalf)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling threesuperior onethird)) (TOKEN_KEY (KEYCODEX AE04) (KEYSYMS 4 dollar EuroSign onequarter)) (TOKEN_KEY (KEYCODEX AE06) (KEYSYMS 6 dead_circumflex NoSymbol onesixth)) (TOKEN_KEY (KEYCODEX AC11) (KEYSYMS dead_acute at apostrophe bar)) (TOKEN_KEY (KEYCODEX TLDE) (KEYSYMS dead_grave notsign bar bar)) (TOKEN_KEY (KEYCODEX BKSL) (KEYSYMS numbersign dead_tilde bar bar)) (TOKEN_KEY (KEYCODEX LSGT) (KEYSYMS backslash bar bar bar)) (TOKEN_INCLUDE “level3(ralt_switch)”))) (SECTION (MAPTYPE (MAPOPTIONS partial alphanumeric_keys xkb_symbols) (MAPNAME “dvorak”)) (MAPMATERIAL (TOKEN_INCLUDE “us(dvorak)”) (TOKEN_NAME Group1 (VALUE “United Kingdom – Dvorak”)) (TOKEN_KEY (KEYCODEX BKSL) (KEYSYMS numbersign asciitilde)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 quotedbl twosuperior NoSymbol)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling threesuperior NoSymbol)) (TOKEN_KEY (KEYCODEX AE04) (KEYSYMS 4 dollar EuroSign NoSymbol)) (TOKEN_KEY (KEYCODEX LSGT) (KEYSYMS backslash bar)) (TOKEN_KEY (KEYCODEX AD01) (KEYSYMS apostrophe at)))) (SECTION (MAPTYPE (MAPOPTIONS partial alphanumeric_keys xkb_symbols) (MAPNAME “mac”)) (MAPMATERIAL (TOKEN_INCLUDE “latin”) (TOKEN_NAME Group1 (VALUE “United Kingdom – Macintosh”)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 at EuroSign)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling numbersign)) (TOKEN_INCLUDE “level3(ralt_switch)”)))

When traversing the tree, we can then pretty-print the layout at wish:

partial default alphanumeric_keys xkb_symbols “basic” {
name[Group1] = “United Kingdom”;
include “latin”
include “level3(ralt_switch_multikey)”
key <AE02> = { [ 2 , quotedbl , twosuperior , oneeighth ] };
key <AE03> = { [ 3 , sterling , threesuperior , sterling ] };
key <AE04> = { [ 4 , dollar , EuroSign , onequarter ] };
key <AC11> = { [ apostrophe , at , dead_circumflex , dead_caron ] };
key <TLDE> = { [ grave , notsign , bar , bar ] };
key <BKSL> = { [ numbersign , asciitilde , dead_grave , dead_breve ] };
key <LSGT> = { [ backslash , bar , bar , brokenbar ] };
… snip …

The code is currently hosted at code.google.com (keyboardlayouteditor) and I intend to move it shortly to FDO.

Improving input method support in GTK+-based apps

When a bug report gets long with many comments, it gets more difficult for someone to get the full picture of what is going on. I’ll attempt to summarise here what’s being said in Bug 321896, Synch gdkkeysyms.h / gtkimcontextsimple.c with X.org 6.9/7.0.

GTK+-based applications use by default the GTK+ Input Method in order to let users type in different languages. Some scripts are very complex (such as SE Asian scripts) and in this case SCIM is used, replacing the GTK+ Input Method. One can even disable GTK+ IM altogether and use the basic X Input Method (XIM) which is provided by the Xorg server, by setting GTK_IM_MODULE to xim. However, the majority of the users have GTK+ IM enabled.

Between GTK+ IM and XIM, the keyboard layouts are being managed by the xkeyboard-config project and Sergey Udaltsov. A keyboard layout is simply a mapping of keyboard keys to Unicode characters, but you can also have compose sequences for some characters using what we call dead keys. When you press a dead key nothing appears on screen but when you press a letter immediately afterwards, you can get an á. This functionality is common to add accents, and there is a big table for these compose sequences (1.3MB) and what Unicode characters they produce.

If you change your keyboard layout (System/Preferences/Keyboard/Layout) to something like U.S. English International (with dead keys), then the ‘ key on your keyboard becomes dead_acute, and the compose sequence

<dead_acute> <a>  : "á"   U00E1 # LATIN SMALL LETTER A WITH ACUTE

works when you press and then a.

There is an issue with compose sequences and input methods; XIM maintains the official upstream version of the compose sequences, and projects such as GTK+ and SCIM carry their own copies of that table.

The issue with GTK+ regarding the compose sequences is that it has a very old version compared to what is available upstream. This is what Bug 321896 is about.

The bug would be have been resolved much much earlier if it wasn’t for the insistence of the GTK+ maintainers to cut the fat and reduce the size of the table (~6000 entries) with clever optimisations.

Tor suggested a clever optimisation; a good number of compose sequences (which looks like <dead_acute> <a> : “á”) resemble the decomposed form (a la Unicode) of those characters. Thus, we can let the user type what she wants, and we can try Unicode normalisation to see if the sequence is composed to a single Unicode character. Lets demonstrate in Python,

$ python
>>> import unicodedata
>>> sequence=[65, 0x301]     # That's 'a' and acute
>>> result = unicodedata.normalize('NFC',"".join(map(unichr, sequence)))
>>> result
>>> print len(result)
>>> print result

That long line above takes the array, applies the unichr() function on each member so that they become Unicode characters and then joins them in a single string. Finally, it normalises the (decomposed) string to a single character. The fact that the resulting string has length 1 (single character) is key to this optimisation. Over 1000 compose sequences can be removed from the compose table through this optimisation. This includes a big chunk of the Latin Unicode blocks, about a few dozens of Cyrillic characters, all of modern Greek and Greek polytonic, some Indic languages (are they actually used?) and other misc sequences.

Matthias laid out the requirements for the optimisation of the remaining compose sequences; ① it has to be static const so a single copy is shared all over the place, ② the first column (out of six) is repeated too often, thus use subtables, and ③ each row ends with a varying number of zeroes, so cut on those zeroes as well. This also required the automatic generation of the optimised table using a script.

The work has not finished yet, and requires testing of the patch. The high priority testing is that keyboard layouts do not get any regressions (that is, compose sequences with dead keys must continue to work along with any new sequences).

With an updated compose table in GTK+, one can write things like ⒼⓃⓄⓂⒺ and all variations of accents on characters, in an easier way.

I’ld like to thank Matthias and Tor for their support in this work. And Jeff for adding this blog to Planet GNOME!


(see http://www.guadec.org/schedule/warmup)

At the first presentation, Quim Gil talked about GNOME marketing, what have been done, what is the goal of marketing. He showed a focused mind on important marketing tasks; it is easy to get carried away and not be effective, a mistake that happens in several projects.

The next session was by Tomas Frydrych (Open Hand – I have their sticker on my laptop!) on memory use in GNOME applications. Many people complain that XYZ is bloated. However, this does not convey what exactly happens; pretty useless. In addition, the common tools that show memory use do not show the proper picture because of the memory management techniques. That is, due to shared libraries, the total memory occupied by an application appears very big. A tool examined is exmap. This tool uses a kernel module that shows memory use of applications by reading in /proc. It takes a snapshot of memory use; it’s not real-time info. It comes with a GTK+ front-end (gexmap) that requires a big screen (oops, PDAs). However, it is not suitable for internet tablets and other low-spec devices. Therefore, they came up with exmap-console which addresses the shortcommings. It has a console interface based on the readline library.

Here are the rest of my notes. Hope they make sense to you.

. exmap –interactive
. ?: help
. Head: quite useful (dynamic allocation)
. Mapped:
. Sole use: memory that app is using on its own (rss?)
. “sort vm”
. “print” or “p”
. “add nautilus”
. “clear”
. “detail file” (what executables/libs loaded and how much consume)
. “detail none”

Sole use
. valgrind, to analyse Sole Use memory?
. “detail ????”

Lots of small libraries: overhead

Looking ahead
. Pagemap: by Matt Macall
. http://projects.o-hand.com/exmap-console/

. Sole use: ~18MB ;-(

Tomas was apparently running Ubuntu with the English UK locale. The English UK translation team is doing an amazing job at the translation stats. Actually, most messages are copied, however with a script one can pick up words such as organization and change to organisation. The problem here is that, for example, the GAIM mo file is 215KB (?), however for the British English translation the actual changes should be less than 2-3KB. Messages that are missing from a translation mean that the original US English messages will be used. I’ll have to find how to use msgfilter to make messages untranslated if msgid == msgstr. Where is Danilo?

After lunch time (did not go for lunch), I went to the Accerciser session. Pretty cool tool, something I have been look for. Accerciser uses the accessibility framework of GNOME in order to inspect the windows of running applications and see into the properties. A good use is to identify if elements such as text boxes come with description labels; they are important to be there for accessibility purposes (screen reader), as a person that depends on software to read (text to speech) the contents of windows.

The next session was GNOME accessibility for blind people. Jan Buchal gave an excellent presentation.

My notes,

. is from Chech republic, is blind himself. has been using computers for 20+ years

. from user perspective
. users, regular and irregular 😉
. software
. firefox 3.0beta – ok for accessibility other versions no
. gaim messenger ok
. openoffice.org ok but did not try
. orca screenreader ^^^ works ok.
. generally ready for prime time
. ubuntu guy for accessibility was there
. made joke about not having/needing display slides ;-]
. synthesizer: festival, espeak, etc – can choose
. availability of voices
. javascript: not good for accessibility
. links/w3m: just fine!
. firefox3 makes accessibility now possible.
. web designer education, things like title=””, alt=”” for images.
. OOo, not installed but should work, ooo-gnome
. “braillcom” company name
. “speech dispatcher”
. logical events
. have short sound event instead of “button”, “input form”
. another special sound for emacs prompt, etc.
. uses emacs
. have all events spoken, such as application crashing.
. problems of accessibility
. not money main factor, but still exists.
. standard developers do not use accessibility functions
. “accessor” talk, can help
. small developer group on accessiblity, may not cooperate well
. non-regular users (such as blind musician)
. musicians
. project “singing computer”
. gtk, did not have good infrastructure
. used lilypond (music typesetter, good but not simple to use)
. singing mode in festival
. use emacs with special mode to write music scores (?)
. write music score and have the computer sing it (this is not “caruso”)
. gnome interface for lilypond would be interesting
. chemistry for blind
. gtk+
. considering it
. must also work, unfortunately, on windows
. gtk+ for windows, not so good for accessibility
. conclusion: free accessibility
. need users so that applications can be improved
. have festival synthesizer, not perfect but usable
. many languages, hindi, finnish, afrikaans
. endinburgh project, to reimplement festival better
. proprietary software is a disadvantage
. q: how do you learn to use new software?
. a: has been a computer user for 20+ years, is not good candidate to say
. a: if you are dedicated, you can bypass hardles, old lady emacs/festival/lilypond
. brrlcom, not for end-users(?)
. developer problem?
. generally there is lack of documentation; easy to teach what a developer needs to know
. so that the application is accessible
. HIG Human Interface Guidelines, accessible to the developers
. “speakup” project
. Willy, from Sun microsystems, working on accessibility for +20 years, Lead of Orca.
. developers: feel accessibility is a hindrance to development
. in practice the gap is not huge
. get tools (glade) and gtk+ to come with accessibility on by default
. accessibility
. is not only for people with disabilities
. can do amazing things like 3d interfaces something

These summaries are an important example of the rule that during presentation, participants tend to remember only about 8% of the material. In some examples, even less is being recollected.

Multimedia support in Ubuntu Linux 6.06

With Ubuntu Linux 6.06, it is much clear how to install those codecs in order to get broad multimedia file support.

In Ubuntu, the multimedia infrastructure is handled by GStreamer; you install GStreamer plugins and any application that uses GStreamer can immediately benefit from the new codec support.

A typical installation of Ubuntu will bring in the free and open-source codecs by default. This includes the base gstreamer plugins package, gstreamer0.10-plugins-base that covers

  1. /usr/lib/gstreamer-0.10/libgstadder.so
  2. /usr/lib/gstreamer-0.10/libgstaudioconvert.so
  3. /usr/lib/gstreamer-0.10/libgstaudiorate.so
  4. /usr/lib/gstreamer-0.10/libgstaudioresample.so
  5. /usr/lib/gstreamer-0.10/libgstaudiotestsrc.so
  6. /usr/lib/gstreamer-0.10/libgstcdparanoia.so
  7. /usr/lib/gstreamer-0.10/libgstdecodebin.so
  8. /usr/lib/gstreamer-0.10/libgstffmpegcolorspace.so
  9. /usr/lib/gstreamer-0.10/libgstogg.so
  10. /usr/lib/gstreamer-0.10/libgstplaybin.so
  11. /usr/lib/gstreamer-0.10/libgstsubparse.so
  12. /usr/lib/gstreamer-0.10/libgsttcp.so
  13. /usr/lib/gstreamer-0.10/libgsttheora.so
  14. /usr/lib/gstreamer-0.10/libgsttypefindfunctions.so
  15. /usr/lib/gstreamer-0.10/libgstvideo4linux.so
  16. /usr/lib/gstreamer-0.10/libgstvideorate.so
  17. /usr/lib/gstreamer-0.10/libgstvideoscale.so
  18. /usr/lib/gstreamer-0.10/libgstvideotestsrc.so
  19. /usr/lib/gstreamer-0.10/libgstvolume.so
  20. /usr/lib/gstreamer-0.10/libgstvorbis.so

With a properly encoded multimedia file, you can play music or video with subtitles. Such good codecs are Ogg, Vorbis and Theora. You can also rip CDs; cdparanoia is also there.
By default you also get the good package, gstreamer0.10-plugins-good
It contains

  1. /usr/lib/gstreamer-0.10/libgst1394.so
  2. /usr/lib/gstreamer-0.10/libgstaasink.so
  3. /usr/lib/gstreamer-0.10/libgstalaw.so
  4. /usr/lib/gstreamer-0.10/libgstalpha.so
  5. /usr/lib/gstreamer-0.10/libgstapetag.so
  6. /usr/lib/gstreamer-0.10/libgstavi.so
  7. /usr/lib/gstreamer-0.10/libgstautodetect.so
  8. /usr/lib/gstreamer-0.10/libgstcacasink.so
  9. /usr/lib/gstreamer-0.10/libgstcdio.so
  10. /usr/lib/gstreamer-0.10/libgsteffectv.so
  11. /usr/lib/gstreamer-0.10/libgstgoom.so
  12. /usr/lib/gstreamer-0.10/libgstid3demux.so
  13. /usr/lib/gstreamer-0.10/libgstlevel.so
  14. /usr/lib/gstreamer-0.10/libgstefence.so
  15. /usr/lib/gstreamer-0.10/libgstmulaw.so
  16. /usr/lib/gstreamer-0.10/libgstossaudio.so
  17. /usr/lib/gstreamer-0.10/libgstrtp.so
  18. /usr/lib/gstreamer-0.10/libgstrtsp.so
  19. /usr/lib/gstreamer-0.10/libgstsmpte.so
  20. /usr/lib/gstreamer-0.10/libgsttaglib.so
  21. /usr/lib/gstreamer-0.10/libgstudp.so
  22. /usr/lib/gstreamer-0.10/libgstvideobox.so
  23. /usr/lib/gstreamer-0.10/libgstvideoflip.so
  24. /usr/lib/gstreamer-0.10/libgstwavenc.so
  25. /usr/lib/gstreamer-0.10/libgstwavparse.so
  26. /usr/lib/gstreamer-0.10/libgstauparse.so
  27. /usr/lib/gstreamer-0.10/libgstdebug.so
  28. /usr/lib/gstreamer-0.10/libgstnavigationtest.so
  29. /usr/lib/gstreamer-0.10/libgstalphacolor.so
  30. /usr/lib/gstreamer-0.10/libgstcairo.so
  31. /usr/lib/gstreamer-0.10/libgstflxdec.so
  32. /usr/lib/gstreamer-0.10/libgstmatroska.so
  33. /usr/lib/gstreamer-0.10/libgstvideomixer.so
  34. /usr/lib/gstreamer-0.10/libgstcutter.so
  35. /usr/lib/gstreamer-0.10/libgstmultipart.so
  36. /usr/lib/gstreamer-0.10/libgstflac.so
  37. /usr/lib/gstreamer-0.10/libgstjpeg.so
  38. /usr/lib/gstreamer-0.10/libgstpng.so
  39. /usr/lib/gstreamer-0.10/libgstspeex.so
  40. /usr/lib/gstreamer-0.10/libgstgconfelements.so
  41. /usr/lib/gstreamer-0.10/libgstshout2.so
  42. /usr/lib/gstreamer-0.10/libgstvideobalance.so
  43. /usr/lib/gstreamer-0.10/libgsticydemux.so
  44. /usr/lib/gstreamer-0.10/libgstximagesrc.so
  45. /usr/lib/gstreamer-0.10/libgstannodex.so
  46. /usr/lib/gstreamer-0.10/libgstgdkpixbuf.so
  47. /usr/lib/gstreamer-0.10/libgsthalelements.so
  48. /usr/lib/gstreamer-0.10/libgstdv.so

This includes generic AVI support, access to digital video and Firewire devices, visualisers, the Matroska codec, access to shoutcast servers, the speex audio codec, the flac codec and many more.

At this point, you can install Pitivi, a gstreamer-enabled video editor written in Python that helps you create your own movie. Make sure you install gstreamer0.10-gnonlin which enables non-linear editing in gstreamer.

Up to here you got free and open-source software.

You can continue with more codecs by installing the package gstreamer0.10-plugins-ugly. This package is not part of the official Ubuntu distribution; you need to enable the Universe repository. Use System/Administration/Synaptic Package Manager to install these additional packages.
Ugly are the plugins and codecs that may have distribution problems in some countries.

Ugly includes

  1. /usr/lib/gstreamer-0.10/libgsta52dec.so
  2. /usr/lib/gstreamer-0.10/libgstasf.so
  3. /usr/lib/gstreamer-0.10/libgstdvdlpcmdec.so
  4. /usr/lib/gstreamer-0.10/libgstdvdread.so
  5. /usr/lib/gstreamer-0.10/libgstdvdsub.so
  6. /usr/lib/gstreamer-0.10/libgstiec958.so
  7. /usr/lib/gstreamer-0.10/libgstmad.so
  8. /usr/lib/gstreamer-0.10/libgstmpeg2dec.so
  9. /usr/lib/gstreamer-0.10/libgstmpegaudioparse.so
  10. /usr/lib/gstreamer-0.10/libgstmpegstream.so
  11. /usr/lib/gstreamer-0.10/libgstrmdemux.so
  12. /usr/lib/gstreamer-0.10/libgstsid.so

This package will bring in, among others, DVD playback and subtitle support, ASF file support, MP3 support (MAD package) and MPEG2 video playback.
You can also get MP3 support if you install the gstreamer0.10-fluendo-mp3 plugin which is available from Universe as well. This package is probably free to use in any country thanks to the efforts of the Fluendo team.

It appears that if you install ugly, it is good to install gstreamer0.10-ffmpeg so that you get support for

FFmpeg plugin for GStreamer

This GStreamer plugin supports a large number of audio and video compression
formats through the use of the FFmpeg library. The plugin contains GStreamer
elements for encoding 40+ formats (MPEG, DivX, MPEG4, AC3, DV, …), decoding
90+ formats
(AVI, MPEG, OGG, Matroska, ASF, …), demuxing 30+ formats, and
colorspace conversion.

Finally, there is a package gstreamer0.10-plugins-bad with plugins of potentially suboptimal quality. It includes

  1. /usr/lib/gstreamer-0.10/libgstbz2.so
  2. /usr/lib/gstreamer-0.10/libgstcdxaparse.so
  3. /usr/lib/gstreamer-0.10/libgstdtsdec.so
  4. /usr/lib/gstreamer-0.10/libgstfreeze.so
  5. /usr/lib/gstreamer-0.10/libgstgsm.so
  6. /usr/lib/gstreamer-0.10/libgstmms.so
  7. /usr/lib/gstreamer-0.10/libgstmodplug.so
  8. /usr/lib/gstreamer-0.10/libgstmusepack.so
  9. /usr/lib/gstreamer-0.10/libgstqtdemux.so
  10. /usr/lib/gstreamer-0.10/libgsttrm.so
  11. /usr/lib/gstreamer-0.10/libgstspeed.so
  12. /usr/lib/gstreamer-0.10/libgstswfdec.so
  13. /usr/lib/gstreamer-0.10/libgsttta.so
  14. /usr/lib/gstreamer-0.10/libgstvideo4linux2.so
  15. /usr/lib/gstreamer-0.10/libgstwavpack.so
  16. /usr/lib/gstreamer-0.10/libgstxingheader.so
  17. /usr/lib/gstreamer-0.10/libgstneonhttpsrc.so

With bad you get GSM audio codec support, MMS support, QT playback support for some formats, Flash (SWF) playing support, Video4Linux2 support, MUSEPACK support and a few more.

Ελληνικά στην αλληλογραφία, μέρος πρώτο

Πρέπει να λαμβάνετε γράμματα / ανακοινώσεις από μερικούς δικτυακούς τόπους όπου η κωδικοποίηση για τα ελληνικά δεν είναι σωστή, είτε στο σώμα του μηνύματος, είτε στην κεφαλίδα (From: “Ανακοίνωση” ).

Συγκεκριμένα, δεν καθορίζεται η κωδικοποίηση οπότε είναι θέμα εξ ορισμού ρυθμίσεων του παραλήπτη για να δει το αποτέλεσμα.

Ας δούμε πως μπορείτε μέσα από μια εφαρμογή PHP να στείλετε αλληλογραφία με ελληνικά. Το ίδιο μπορεί να γίνει και από άλλες γλώσσες, όπως Perl και Python.


$from = “From: \”” . mb_encode_mimeheader(‘Όνομα Αποστολέα’) . “\” < αποστολέας στο gmail τελεία com>“;
$to = mb_encode_mimeheader(‘Όνομα Παραλήπτη’) . ” < παραλήπτης στο gmail τελεία com>“;
$subject = ‘Θέμα γράμματος’;
$body = ‘Περιεχόμενο του γράμματος.’;

mb_send_mail($to, $subject, $body, $from);

Το γράμμα που θα παραχθεί θα μοιάζει με

Από: Όνομα Αποστολέα < αποστολέας στο gmail τελεία com>
Προς: Όνομα Παραλήπτη < παραλήπτης στο gmail τελεία com>
Θέμα: Θέμα γράμματος

Περιεχόμενο του γράμματος.

Απαιτεί την εγκατάσταση του πακέτου php-mbstring που το έχουν όλες οι καλές διανομές Linux. Διαφορετικά είναι δυνατόν
να έχετε το ίδιο αποτέλεσμα αλλά θα κάνετε τα παραπάνω χειρωνακτικά.

Ακόμα, πρέπει να ρυθμίσετε το /etc/php.ini με τα παρακάτω:

; language for internal character representation.
; Neutral σημαίνει Unicode
mbstring.language = Neutral

; internal/script encoding.
; Some encoding cannot work as internal encoding.
; (e.g. SJIS, BIG5, ISO-2022-*)
mbstring.internal_encoding = UTF-8

; http input encoding.
mbstring.http_input = UTF-8

; http output encoding. mb_output_handler must be
; registered as output buffer to function
mbstring.http_output = UTF-8

; enable automatic encoding translation accoding to
; mbstring.internal_encoding setting. Input chars are
; converted to internal encoding by setting this to On.
; Note: Do _not_ use automatic encoding translation for
; portable libs/applications.
mbstring.encoding_translation = On

; substitute_character used when character cannot be converted
; one from another
; σημαίνει ότι στην μετατροπή αν κάτι πάει στραβά, θα εκτυπώσει των κωδικό U+xxxx του χαρακτήρα.
mbstring.substitute_character = long;

Αν είστε χρήστης της εφαρμογής phplist, ενημερώστε τη σελίδα αυτή.

Σημείωση: Όλα τα παραπάνω είναι σε κωδικοποίηση utf-8 (Unicode).