Difference between revisions of "Information for Developers"

From OpenPLi Wiki
Jump to: navigation, search
(Create your own build)
 
(26 intermediate revisions by 7 users not shown)
Line 19: Line 19:
 
Install prerequisite packages, as described here: [http://www.openembedded.org/wiki/OEandYourDistro<span style="color:SteelBlue">http://www.openembedded.org/wiki/OEandYourDistro</span>]
 
Install prerequisite packages, as described here: [http://www.openembedded.org/wiki/OEandYourDistro<span style="color:SteelBlue">http://www.openembedded.org/wiki/OEandYourDistro</span>]
  
For Ubuntu, that's:
+
For Ubuntu 22.04 LTS, that's:
  
 
   sudo apt-get install sed wget cvs subversion git-core \
 
   sudo apt-get install sed wget cvs subversion git-core \
 
   coreutils unzip texi2html texinfo docbook-utils \
 
   coreutils unzip texi2html texinfo docbook-utils \
   gawk python-pysqlite2 diffstat help2man make gcc build-essential g++ \
+
   gawk python3-distutils diffstat help2man make gcc build-essential g++ \
   desktop-file-utils chrpath default-jre gettext zip libssl-dev
+
   desktop-file-utils chrpath default-jre gettext zip libssl-dev \
 +
  zstd liblz4-tool
 +
 
 +
You may need to create a symlink makeinfo->texi2any (in the bin folder) and a symlink makeinfo->/bin/texi2any in the hosttools folder.
  
 
For CentOS, that's:
 
For CentOS, that's:
Line 30: Line 33:
 
   sudo yum install gawk make wget tar bzip2 gzip python unzip perl patch \
 
   sudo yum install gawk make wget tar bzip2 gzip python unzip perl patch \
 
     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \
 
     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \
     openssl-devel zip python3 perl-Thread-Queue
+
     openssl-devel zip python3 perl-Thread-Queue libstdc++-static \
 +
    glibc-devel.i686 libgcc.i686 libstdc++-devel.i686
 +
 
 +
Setup the build environment:
 +
sudo chmod ugo+rw /opt
 +
mkdir -p /opt/openpli
 +
cd /opt/openpli
  
 
Clone the openpli repository:
 
Clone the openpli repository:
  git clone git@github.com:OpenPLi/openpli-oe-core.git
+
git clone https://github.com/OpenPLi/openpli-oe-core.git
 +
or
 +
git clone git@github.com:OpenPLi/openpli-oe-core.git
  
Setup the environment
+
Setup the OpenPli environment
 
   cd openpli-oe-core
 
   cd openpli-oe-core
 
+
   make
Select the branch to build from
 
   git checkout develop
 
 
 
Install the required submodules
 
  git submodule update --init --recursive
 
  
 
Build your first image
 
Build your first image
 
   MACHINE=xxXXXX make image
 
   MACHINE=xxXXXX make image
 +
  or
 +
  cd build
 +
  MACHINE=xxXXXX
 +
  source env.source
 +
  bitbake -k openpli-enigma2-image
  
 
To find the valid MACHINE=xxXXXX you can have a look at the .conf file in the meta-folder of the manufacturer MMMMM:
 
To find the valid MACHINE=xxXXXX you can have a look at the .conf file in the meta-folder of the manufacturer MMMMM:
 
   cd meta-MMMMM/conf/machine
 
   cd meta-MMMMM/conf/machine
 
   ls
 
   ls
 +
 +
If the build is successfull the resulting file will be into:
 +
 +
  ./openpli-oe-core/build/tmp/deploy/images/xxXXXX/
 +
  openpli-homebuild-xxXXXX_usb.zip
 +
 +
 +
===Updating===
 +
Before every build, update your cloned git repositories with
 +
cd openpli-oe-core
 +
make update
  
 
Note that by default, the develop branch of Enigma is used when you build an image, assuming that most "homebuild" users will want to build the latest development image.
 
Note that by default, the develop branch of Enigma is used when you build an image, assuming that most "homebuild" users will want to build the latest development image.
  
If you want to build a release image, you need to specifcy the correct branch when building:
+
If you want to build a homebuild release image, you need to specify the correct branch when building:
 
   cd openpli-oe-core
 
   cd openpli-oe-core
 
   git checkout release-7.2
 
   git checkout release-7.2
Line 79: Line 101:
 
   rpm2cpio tar-1.28-3.1.mga5.x86_64.rpm | cpio -idcmv
 
   rpm2cpio tar-1.28-3.1.mga5.x86_64.rpm | cpio -idcmv
 
   cp /tmp/usr/bin/tar /usr/bin
 
   cp /tmp/usr/bin/tar /usr/bin
 +
 +
=== Ubuntu incompatibilities ===
 +
 +
When running a build on ubuntu, you can run into the issue that the build is aborted by the sanity checker, complaining that "makeinfo" does not exist. Even though the package texinfo is installed. This is caused by the fact makeinfo is now a symlink, which doesn't make it into hosttools.
 +
 +
To fix this, execute
 +
  cd <your-build-dir>/build/tmp/hosttools
 +
  ln -s /usr/bin/texi2any makeinfo
 +
 +
=== Distro's with Python 3 older than 3.9 ===
 +
 +
Examples of distro's are RHEL 8 (and clones), or Ubuntu 18, both using Python 3.6. When using these, you can get build errors when python scripts are called directory from a recipe or a makefile, stating
 +
  No module named '_sysconfigdata'
 +
 +
You can work around this problem by finding the Python package directory (in RHEL it is /lib64/python3.6, in Ubuntu it is /usr/lib/python3.6), look for either "_sysconfigdata__linux_x86_64-linux-gnu.py" or "_sysconfigdata_m_linux_x86_64-linux-gnu.py", and symlink it it "_sysconfigdata.py"
 +
 +
=== Cleaning up the sstate cache ===
 +
 +
You will notice that after each build, the storage used by the sstate-cache will increase. This is because this cache holds all data required to be able to skip building packages that haven't changed, and previous release data is not purged by bitbake.
 +
 +
To address this, make your git directory the current directory, and run
 +
  ./openembedded-core/scripts/sstate-cache-management.sh --cache-dir=./build/sstate-cache --stamps-dir=./build/tmp/stamps --yes
 +
 +
Note that if your cache is old, this may take a long time, as on our buildserver (which does build all images!) the result after a second build run is "1385449 out of 1682374 files will be removed!"
  
 
=='''Your own feed server'''==
 
=='''Your own feed server'''==
Line 90: Line 136:
 
As an example, if you've installed apache on Ubuntu, it will share /var/www/html/ with the world. Create a feed subdirectory, and then add a symlink to the "ipk" folder of your build, for example:
 
As an example, if you've installed apache on Ubuntu, it will share /var/www/html/ with the world. Create a feed subdirectory, and then add a symlink to the "ipk" folder of your build, for example:
  
   sudo mkdir /var/www/html/feed
+
   sudo mkdir /var/www/html/feeds
   sudo ln -s ${HOME}/work/openpli-dev/build/tmp/deploy/ipk /var/www/html/feed/openpli-dev
+
   sudo ln -s ${HOME}/work/openpli-oe-core/build/tmp/deploy/ipk/ /var/www/html/feeds/openpli-homebuild
  
 
To tell your box about this feed location, edit your local.conf or site.conf to contain the following lines:
 
To tell your box about this feed location, edit your local.conf or site.conf to contain the following lines:
  
   FEED_NAME = "openpli-dev"
+
   FEED_NAME = "openpli-homebuild"
 
   DISTRO_HOST = "mybuildpc.local"
 
   DISTRO_HOST = "mybuildpc.local"
  
Line 105: Line 151:
  
 
Using PHP 5.4+:
 
Using PHP 5.4+:
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   php -S 0.0.0.0:80
 
   php -S 0.0.0.0:80
  
 
Using Python 2.x:
 
Using Python 2.x:
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   python -m SimpleHTTPServer 80
 
   python -m SimpleHTTPServer 80
  
 
Using Python 3.x:
 
Using Python 3.x:
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   python3 -m http.server 80
 
   python3 -m http.server 80
  
 
Using Ruby:
 
Using Ruby:
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   ruby -rwebrick -e'WEBrick::HTTPServer.new(:Port => 80, :DocumentRoot => Dir.pwd).start'
 
   ruby -rwebrick -e'WEBrick::HTTPServer.new(:Port => 80, :DocumentRoot => Dir.pwd).start'
  
 
Using Ruby 1.9.2+:
 
Using Ruby 1.9.2+:
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   ruby -run -ehttpd . -p80
 
   ruby -run -ehttpd . -p80
  
 
Using Perl:
 
Using Perl:
 
   cpan HTTP::Server::Brick # one-time install of perl dependency
 
   cpan HTTP::Server::Brick # one-time install of perl dependency
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   perl -MHTTP::Server::Brick -e '$s=HTTP::Server::Brick->new(port=>80); $s->mount("/"=>{path=>"."}); $s->start'
 
   perl -MHTTP::Server::Brick -e '$s=HTTP::Server::Brick->new(port=>80); $s->mount("/"=>{path=>"."}); $s->start'
  
 
Using Busybox HTTPd:
 
Using Busybox HTTPd:
   cd /var/www/html/feed
+
   cd /var/www/html/feeds
 
   busybox httpd -f -p 80
 
   busybox httpd -f -p 80
  

Latest revision as of 19:21, 18 March 2024

'I'm a developer, is there any technical info available from the PLi® team?

Welcome and tell your developer friends about us

Yes and welcome aboard! If you have specific question, you can use our PLi® Third Party Development forum to ask it. You will find we are most accommodating.

This page is about software development using OpenEmbedded-core. If you're looking for the text that used to be on the this page, it's now called developer-information-old.


Create your own build

See also this forum post on setting up a development environment: http://openpli.org/forums/topic/18806-openpli-quick-setup-ubuntudebian/

Basically, it boils down to this:

Run Linux. Most of us use the latest Ubuntu desktop release, I suggest you do the same, if you don't know what to pick.

Make sure the disk you use for the OE tree supports symbolic links, and you use at least git v1.8.0.

Install prerequisite packages, as described here: http://www.openembedded.org/wiki/OEandYourDistro

For Ubuntu 22.04 LTS, that's:

 sudo apt-get install sed wget cvs subversion git-core \
  coreutils unzip texi2html texinfo docbook-utils \
  gawk python3-distutils diffstat help2man make gcc build-essential g++ \
  desktop-file-utils chrpath default-jre gettext zip libssl-dev \
  zstd liblz4-tool

You may need to create a symlink makeinfo->texi2any (in the bin folder) and a symlink makeinfo->/bin/texi2any in the hosttools folder.

For CentOS, that's:

 sudo yum install gawk make wget tar bzip2 gzip python unzip perl patch \
    diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \
    openssl-devel zip python3 perl-Thread-Queue libstdc++-static \
    glibc-devel.i686 libgcc.i686 libstdc++-devel.i686

Setup the build environment:

sudo chmod ugo+rw /opt
mkdir -p /opt/openpli
cd /opt/openpli

Clone the openpli repository:

git clone https://github.com/OpenPLi/openpli-oe-core.git
or
git clone git@github.com:OpenPLi/openpli-oe-core.git

Setup the OpenPli environment

 cd openpli-oe-core
 make

Build your first image

 MACHINE=xxXXXX make image
 or
 cd build
 MACHINE=xxXXXX
 source env.source
 bitbake -k openpli-enigma2-image

To find the valid MACHINE=xxXXXX you can have a look at the .conf file in the meta-folder of the manufacturer MMMMM:

 cd meta-MMMMM/conf/machine
 ls

If the build is successfull the resulting file will be into:

 ./openpli-oe-core/build/tmp/deploy/images/xxXXXX/
 openpli-homebuild-xxXXXX_usb.zip


Updating

Before every build, update your cloned git repositories with

cd openpli-oe-core
make update

Note that by default, the develop branch of Enigma is used when you build an image, assuming that most "homebuild" users will want to build the latest development image.

If you want to build a homebuild release image, you need to specify the correct branch when building:

 cd openpli-oe-core
 git checkout release-7.2
 git submodule update --init --recursive
 ENIGMA2_BRANCH=release-7.2 MACHINE=xxXXXX make image

The resulting file will be into:

 ./openpli-oe-core/build/tmp/deploy/images/xxXXXX/
 openpli-homebuild-xxXXXX_usb.zip

CentOS incompatibilities

Note that CentOS, even CentOS 7, comes with gcc 4, which is pretty old. It is advised to use DevToolsets to switch to 7 or 8:

 sudo yum install centos-release-scl devtoolset-7 devtoolset-8

which gives you the option to use versions 6, 7 and 8 as well. You can enable a version using "scl enable devtoolset-8".

When you want to compile OpenPLi-8+ (i.e. an image based on Yocto Zeus or higher), you need at least tar v1.28, but CentOS comes with 1.26, and bitbake refuses to run.

You can address this issue by (as root):

 cd /tmp
 wget https://rpmfind.net/linux/mageia/distrib/6/x86_64/media/core/release/tar-1.28-3.1.mga5.x86_64.rpm
 rpm2cpio tar-1.28-3.1.mga5.x86_64.rpm | cpio -idcmv
 cp /tmp/usr/bin/tar /usr/bin

Ubuntu incompatibilities

When running a build on ubuntu, you can run into the issue that the build is aborted by the sanity checker, complaining that "makeinfo" does not exist. Even though the package texinfo is installed. This is caused by the fact makeinfo is now a symlink, which doesn't make it into hosttools.

To fix this, execute

 cd <your-build-dir>/build/tmp/hosttools
 ln -s /usr/bin/texi2any makeinfo

Distro's with Python 3 older than 3.9

Examples of distro's are RHEL 8 (and clones), or Ubuntu 18, both using Python 3.6. When using these, you can get build errors when python scripts are called directory from a recipe or a makefile, stating

 No module named '_sysconfigdata'

You can work around this problem by finding the Python package directory (in RHEL it is /lib64/python3.6, in Ubuntu it is /usr/lib/python3.6), look for either "_sysconfigdata__linux_x86_64-linux-gnu.py" or "_sysconfigdata_m_linux_x86_64-linux-gnu.py", and symlink it it "_sysconfigdata.py"

Cleaning up the sstate cache

You will notice that after each build, the storage used by the sstate-cache will increase. This is because this cache holds all data required to be able to skip building packages that haven't changed, and previous release data is not purged by bitbake.

To address this, make your git directory the current directory, and run

 ./openembedded-core/scripts/sstate-cache-management.sh --cache-dir=./build/sstate-cache --stamps-dir=./build/tmp/stamps --yes

Note that if your cache is old, this may take a long time, as on our buildserver (which does build all images!) the result after a second build run is "1385449 out of 1682374 files will be removed!"

Your own feed server

Once you've built your own image, you'd want to keep it up to date. You can just let the box update itself from your build PC using the GUI as if running a full distro. To do that, you first need to build the optional packages that go into the feed:

 cd openpli-oe-core
 MACHINE=xxXXXX make feed

Next, you need to install a webserver of some sort on the build PC, for example Apache2.

As an example, if you've installed apache on Ubuntu, it will share /var/www/html/ with the world. Create a feed subdirectory, and then add a symlink to the "ipk" folder of your build, for example:

 sudo mkdir /var/www/html/feeds
 sudo ln -s ${HOME}/work/openpli-oe-core/build/tmp/deploy/ipk/ /var/www/html/feeds/openpli-homebuild

To tell your box about this feed location, edit your local.conf or site.conf to contain the following lines:

 FEED_NAME = "openpli-homebuild"
 DISTRO_HOST = "mybuildpc.local"

After this, if you run a new image build, it will make the /etc/opkg/ files on the box point to your feed. After installing the built image on your box, you should be able to upgrade with opkg commands or the enigma2 GUI.

No real webserver handy?

If you don't have a real webserver, like Apache or Nginx, handy, there are some other options available to serve files via http.

Using PHP 5.4+:

 cd /var/www/html/feeds
 php -S 0.0.0.0:80

Using Python 2.x:

 cd /var/www/html/feeds
 python -m SimpleHTTPServer 80

Using Python 3.x:

 cd /var/www/html/feeds
 python3 -m http.server 80

Using Ruby:

 cd /var/www/html/feeds
 ruby -rwebrick -e'WEBrick::HTTPServer.new(:Port => 80, :DocumentRoot => Dir.pwd).start'

Using Ruby 1.9.2+:

 cd /var/www/html/feeds
 ruby -run -ehttpd . -p80

Using Perl:

 cpan HTTP::Server::Brick # one-time install of perl dependency
 cd /var/www/html/feeds
 perl -MHTTP::Server::Brick -e '$s=HTTP::Server::Brick->new(port=>80); $s->mount("/"=>{path=>"."}); $s->start'

Using Busybox HTTPd:

 cd /var/www/html/feeds
 busybox httpd -f -p 80

NB: For ports below 1024, like the standard HTTP port 80 in the example above, you need root or administrator privileges.

OpenPLi - Git commands

Here is the link to some basic git commands.