Tag : stackoverflow

post image

How to create a snap with snapcraft for howdoi (CLI utilility for stackoverflow)

In the tutorial How to create a snap for how2 (stackoverflow from the terminal) in Ubuntu 16.04 we saw how to create a snap with snapcraft for the CLI utility called how2. That was a software based on nodejs.

In this post we will repeat the process for another CLI utility called howdoi by Benjamin Gleitzman, which does a similar task with how2 but is implemented in Python and has a few usability differences as well. howdoi does not have yet a package in the repositories of Ubuntu, either.

Since we covered already the details in How to create a snap for how2 (stackoverflow from the terminal) in Ubuntu 16.04, this post would be more focused, and shorter. 🙂

Planning

Reading through https://github.com/gleitz/howdoi we see that howdoi

  1. is software based on Python (therefore: plugin: python)
  2. requires networking (therefore: plugs: [network])
  3. and has no need to save files (therefore it does not need access to the filesystem)

Crafting with snapcraft

Let’s start with snapcraft.

$ mkdir howdoi
$ cd howdoi/
$ snapcraft init
Created snap/snapcraft.yaml.
Edit the file to your liking or run `snapcraft` to get started

Now we edit snap/snapcraft.yaml and here are our changes (in bold) from the initial generated file.

$ cat snap/snapcraft.yaml 
name: howdoi # you probably want to 'snapcraft register <name>'
version: '20170207' # just for humans, typically '1.2+git' or '1.3.2'
summary: instant coding answers via the command line # 79 char long summary
description: |
  Are you a hack programmer? Do you find yourself constantly Googling 
  for how to do basic programing tasks?
  Suppose you want to know how to format a date in bash. Why open your browser 
  and read through blogs (risking major distraction) when you can simply 
  stay in the console and ask howdoi.

grade: stable # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

apps:
  howdoi:
    command: howdoi
    plugs: [network]

parts:
  howdoi:
    plugin: python
    source: https://github.com/gleitz/howdoi.git

First, we selected to use the name howdoi because again, it’s not a reserved name :-). Also, we registered it with snapcraft,

$ snapcraft register howdoi
Registering howdoi.
Congratulations! You're now the publisher for 'howdoi'.

Second, we did not notice a particular branch or tag for howdoi, therefore we put the date of the snap creation.

Third, the summary and the description are just pasted from the Readme.md of the howdoi repository.

Fourth, we select the grade stable and enforce the strict confinement.

The apps: howdoi: command: howdoi is the standard sequence to specify the command that will be exposed to the user. The user will be typing howdoi and the command howdoi inside the snap will be invoked.

The parts: howdoi: plugin: python source: … is the standard sequence to specify that the howdoi that was referenced just earlier, is software written in Python and the source comes from this github repository.

Let’s craft the snap.

$ snapcraft 
Preparing to pull howdoi 
...                                                                     
Pulling howdoi 
...
Preparing to build howdoi 
Building howdoi 
...
Successfully built howdoi
...
Installing collected packages: howdoi, cssselect, Pygments, requests, lxml, pyquery, requests-cache
Successfully installed Pygments-2.2.0 cssselect-1.0.1 howdoi-1.1.9 lxml-3.7.2 pyquery-1.2.17 requests-2.13.0 requests-cache-0.4.13
Staging howdoi 
Priming howdoi 
Snapping 'howdoi' |                                                                       
Snapped howdoi_20170207_amd64.snap
$ snap install howdoi_20170207_amd64.snap --dangerous
howdoi 20170207 installed
$ howdoi format date bash
DATE=`date +%Y-%m-%d`
$ _

Beautiful! It worked!

Publish to the Ubuntu Store

Let’s publish the snap to the Ubuntu Store. We are going to push the file howdoi_20170207_amd64.snap and then check that it has passed the automatic checking. Once it has done so, we release to the stable channel.

$ snapcraft push howdoi_20170207_amd64.snap 
Pushing 'howdoi_20170207_amd64.snap' to the store.
Uploading howdoi_20170207_amd64.snap [=============================================================] 100%
Ready to release!|                                                                                       
Revision 1 of 'howdoi' created.

Just a reminder: We can release the snap to the stable channel simply by running snapcraft release howdoi 1 stable. The alternative to this command, is to do all the following through the Web.

We log in into https://myapps.developer.ubuntu.com/ to check whether snap is ready to publish. In the following screenshots, you would click where the arrows are showing. See the captions for explanations.

Here is the uploaded snap in our account page in the Ubuntu Store. The snap was uploaded using snapcraft, although it is also possible to uploaded from the account page as well.

 

The package (the snap) is ready to publish, because it passed the automated tests and was not flagged for manual review.

By default, the package has not been released to a channel. We click on Release in order to select which channels to release it to.

For this specific package, we select the stable channel. It is not necessary to select the other channels, because by default a higher channel implies those below. Then, click on the Release button.

The package got released, and it shown it got released in stable, candidate, beta and edge (we selected stable, but the rest are implied because “stable” beats the rest.) Note that the Package status has changed to “Published”, and we have the option to Unpublish or even Make private. Ignore the arrow, it was pasted by mistake.

post image

How to create a snap for how2 (stackoverflow from the terminal) in Ubuntu 16.04

Stackoverflow is an invaluable resource for questions related to programming and other subjects.

Normally, the workflow for searching http://stackoverflow.com/, is to search Google using a Web browser. Most probably, the result will be a question from stackoverflow.

A more convenient way to query StackOverflow, is to use the how2 command-line utility.

Here is how it looks:

In this HowTo, we will see:

  1. How to set up snapcraft in order to make the snap
  2. How to write the initial snapcraft.yaml configure
  3. Build the snap with trial and error
  4. Create the final snap
  5. Make the snap available to the Ubuntu Store

Set up snapcraft

snapcraft is a utility that helps us create snaps. Let’s install snapcraft.

$ sudo apt update
...
Reading state information... Done
All packages are up to date.
$ sudo apt install snapcraft
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  snapcraft
...
Preparing to unpack .../snapcraft_2.26_all.deb ...
Unpacking snapcraft (2.26) ...
Setting up snapcraft (2.26) ...
$_

In Ubuntu 16.04, snapcraft was updated in early February and has a few differences from the previous version. Make sure you have snapcraft 2.26 or newer.

Let’s create a new directory for the development of the httpstat snap and initialize it with snapcraft so that create the necessary initial files.

$ mkdir how2
$ cd how2/
$ snapcraft init
Created snap/snapcraft.yaml.
Edit the file to your liking or run `snapcraft` to get started
$ ls -l
total 4
drwxrwxr-x 2 myusername myusername 4096 Feb   6 14:09 snap
$ ls -l snap/
total 4
-rw-rw-r-- 1 myusername myusername 676 Feb   6 14:09 snapcraft.yaml
$ _

We are in this how2/ directory and from here we run snapcraft in order to create the snap. snapcraft will take the instructions from snap/snapcraft.yaml and do its best to create the snap.

These are the initial contents of snap/snapcraft.yaml:

name: my-snap-name # you probably want to 'snapcraft register <name>'
version: '0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: |
  This is my-snap's description. You have a paragraph or two to tell the
  most important story about your snap. Keep it under 100 words though,
  we live in tweetspace and your description wants to look good in the snap
  store.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: devmode # use 'strict' once you have the right plugs and slots

parts:
  my-part:
    # See 'snapcraft plugins'
    plugin: nil

I have formatted as italics the first chunk of configuration lines of snapcraft.yaml, because this chunk is what rarely changes when you develop the snap. The other chunk is the one that the actual actions take place. It is good to distinguish those two chunks.

This snap/snapcraft.yaml configuration file is actually usable and can create an (empty) snap. Let’s create this empty snap, install it, uninstall it and then clean up to the initial pristine state.

$ snapcraft 
Preparing to pull my-part 
Pulling my-part 
Preparing to build my-part 
Building my-part 
Staging my-part 
Priming my-part 
Snapping 'my-snap-name' |                                                                 
Snapped my-snap-name_0.1_amd64.snap
$ snap install my-snap-name_0.1_amd64.snap 
error: cannot find signatures with metadata for snap "my-snap-name_0.1_amd64.snap"
$ snap install my-snap-name_0.1_amd64.snap --dangerous
error: cannot perform the following tasks:
- Mount snap "my-snap-name" (unset) (snap "my-snap-name" requires devmode or confinement override)
Exit 1
$ snap install my-snap-name_0.1_amd64.snap --dangerous --devmode
my-snap-name 0.1 installed
$ snap remove my-snap-name
my-snap-name removed
$ snapcraft clean
Cleaning up priming area
Cleaning up staging area
Cleaning up parts directory
$ ls
my-snap-name_0.1_amd64.snap  snap/
$ rm my-snap-name_0.1_amd64.snap 
rm: remove regular file 'my-snap-name_0.1_amd64.snap'? y
removed 'my-snap-name_0.1_amd64.snap'
$ _

While developing the snap, we will be going through this cycle of creating the snap, testing it and then removing it. There are ways to optimize a bit this process, learn soon we will.

In order to install the snap from a .snap file, we had to use –dangerous because the snap has not been digitally signed. We also had to use –devmode because snapcraft.yaml specifies the developer mode, which is a relaxed (in terms of permissions) development mode.

Writing the snapcraft.yaml for how2

Here is the first chunk of snapcraft.yaml, the chunk that does not change while developing the snap.

name: how2 # you probably want to 'snapcraft register <name>'
version: '20170206' # just for humans, typically '1.2+git' or '1.3.2'
summary: how2, stackoverflow from the terminal
description: |
  how2 finds the simplest way to do something in a unix shell. 
  It is like the man command, but you can query it in natural language.

grade: stable # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

We specify the name and version of the snap. The name is not registered already and it is not reserved, because

$ snapcraft register how2
Registering how2.
Congratulations! You're now the publisher for 'how2'.

We add a suitable summary and description that was copied conveniently from the development page of how2.

We set the grade to stable so that the snap can make it to the stable channel and be available to anyone.

We set the confinement to strict, which means that by default the snap will have no special access (no filesystem access, no network access, etc) unless we carefully allow what is really needed.

Here goes the other chunk.

apps:
  how2:
    command: how2

parts:
  how2:
    plugin: nodejs
    source: https://github.com/santinic/how2.git

How did we write this other chunk?

The apps: how2 : command: how2 is generic. That is, we specify an app that we name as how2, and it is invoked as a command with the name how2. The command could also be bin/how2 or node how2. We will figure out later whether we need to change it because snapcraft will show an error message.

The parts: how2: plugin: nodejs is also generic. We know that how2 is build on nodejs and we figured that one out from the github page of how2. Then, we looked into the list of plugins for snapcraft, and found the nodejs plugin page. At the end of the nodejs plugin page there is a link to examples for the user of nodejs in snapcraft.yaml. This link is actually a search in github with search terms filename:snapcraft.yaml “plugin: nodejs”(in all files that are named snapcraft.yaml, search for “plugin: nodejs”). For this search to work, you need to be logged in to Github first. For the specific case of nodejs, we can try without additional parameters as most examples do not show a use of special parameters.

Work on the snapcraft.yaml with trial and error

We come up with the following snapcraft.yaml by piecing together the chunks from the previous section:

$ cat snap/snapcraft.yamlname: how2 # you probably want to 'snapcraft register <name>'
version: '20170206' # just for humans, typically '1.2+git' or '1.3.2'
summary: how2, stackoverflow from the terminal
description: |
  how2 finds the simplest way to do something in a unix shell. 
  It is like the man command, but you can query it in natural language.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

apps:
  how2:
    command: how2
    plugs:
      - network

parts:
  how2:
    plugin: nodejs
    source: https://github.com/santinic/how2.git

Let’s run snapcraft in order to build the snap.

$ snapcraft clean
Cleaning up priming area
Cleaning up staging area
Cleaning up parts directory
$ snapcraft 
Preparing to pull how2 
Pulling how2 
...
Downloading 'node-v4.4.4-linux-x64.tar.gz'[===============================] 100%
npm --cache-min=Infinity install
...
npm-latest@1.0.2 node_modules/npm-latest
├── vcsurl@0.1.1
├── colors@0.6.2
└── registry-url@3.1.0 (rc@1.1.6)
...
Preparing to build how2 
Building how2 
...
Staging how2 
Priming how2 
Snapping 'how2' |                                                                              
Snapped how2_20170206_amd64.snap
$ _

Wow, it created successfully the snap on the first try! Let’s install it and then test it.

$ sudo snap install how2_20170206_amd64.snap --dangerous
how2 20170206 installed
$ how2 read file while changing
/Cannot connect to Google.
Error: Error on response:Error: getaddrinfo EAI_AGAIN www.google.com:443 : undefined
$ _

It works again, and the only problem is the confinement. We need to allow the snap to access the Internet, and only the Internet.

Add the ability to access the Internet

To be able to access the network, we need to relax the confinement of the snap and allow access to the network interface.

There is an identifier called plugs, and accepts an array of names of interfaces, from the list of available interfaces.

In snapcraft.yaml, you can specify such an array in either of the following formats:

plugs: [ network]
         or
plugs: 
   - network

Here is the final version of snapcraft.yaml for how2:

name: how2 # you probably want to 'snapcraft register <name>'
version: '20170206' # just for humans, typically '1.2+git' or '1.3.2'
summary: how2, stackoverflow from the terminal
description: |
  how2 finds the simplest way to do something in a unix shell. 
  It is like the man command, but you can query it in natural language.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

apps:
  how2:
    command: how2
    plugs: [ network ]

parts:
  how2:
    plugin: nodejs
    source: https://github.com/santinic/how2.git

Let’s create the snap, install and run the test query.

$ snapcraft 
Skipping pull how2 (already ran)
Skipping build how2 (already ran)
Skipping stage how2 (already ran)
Skipping prime how2 (already ran)
Snapping 'how2' |                                                                              
Snapped how2_20170206_amd64.snap
$ sudo snap install how2_20170206_amd64.snap --dangerous
how2 20170206 installed
$ how2 read file while changing
terminal - Output file contents while they change

You can use tail command with -f  :


   tail -f /var/log/syslog 

It's good solution for real time  show.


Press SPACE for more choices, any other key to quit.

That’s it! It works fine!

Make the snap available in the Ubuntu Store

The command snapcraft push will upload the .snap file to the Ubuntu Store. Then, we use the snapcraft release command to release the snap into the beta channel of the Ubuntu Store. Because we specified the grade as devel, we cannot release to the stable channel. When we release a snap to the beta channel, it is considered as released to the edge channel as well (because beta is higher than edge).

$ snapcraft push how2_20170206_amd64.snap 
Pushing 'how2_20170206_amd64.snap' to the store.
Uploading how2_20170206_amd64.snap [====================================================================] 100%
Ready to release!|                                                                                            
Revision 1 of 'how2' created.
$ snapcraft release how2 1 stable
Revision 1 (strict) cannot target a stable channel (stable, grade: devel)
$ snapcraft release how2 1 beta
The 'beta' channel is now open.

Channel    Version    Revision
stable     -          -
candidate  -          -
beta       20170206   1
edge       ^          ^
$ _

Everything looks fine now. Let’s remove the manually-installed snap and install it from the Ubuntu Store.

$ snap remove how2
how2 removed
$ snap info how2
name:      how2
summary:   "how2, stackoverflow from the terminal"
publisher: simosx
description: |
  how2 finds the simplest way to do something in a unix shell. 
  It is like the man command, but you can query it in natural language.
  
channels:              
  beta:   20170206 (1) 11MB -
  edge:   20170206 (1) 11MB -

$ snap install how2
error: cannot install "how2": snap not found
$ snap install how2 --channel=beta
how2 (beta) 20170206 from 'simosx' installed
$ how2 how to edit an XML file
How to change values in XML file

Using XMLStarlet (http://xmlstar.sourceforge.net/):
...omitted...