Difference between revisions of "Information for Developers"
(→Cleaning up the sstate cache) |
(added git-lfs as host dependency) |
||
(26 intermediate revisions by 5 users not shown) | |||
Line 7: | Line 7: | ||
− | + | == | |
=='''Create your own build'''== | =='''Create your own build'''== | ||
See also this forum post on setting up a development environment: [http://openpli.org/forums/topic/18806-openpli-quick-setup-ubuntudebian/<span style="color:SteelBlue">http://openpli.org/forums/topic/18806-openpli-quick-setup-ubuntudebian/</span>] | See also this forum post on setting up a development environment: [http://openpli.org/forums/topic/18806-openpli-quick-setup-ubuntudebian/<span style="color:SteelBlue">http://openpli.org/forums/topic/18806-openpli-quick-setup-ubuntudebian/</span>] | ||
Line 13: | Line 13: | ||
Basically, it boils down to this: | Basically, it boils down to this: | ||
− | Run Linux. Most of us use | + | Run Linux. Most of us use either Ubuntu 22.04 or an EL v9 distro, I suggest you do the same, if you don't know what to pick. |
+ | Stay away from bleeding edge distro's like Fedora, you'll run into all sorts of misery. | ||
− | Make sure the disk you use for the OE tree supports symbolic links, and you use at least git v1.8.0. | + | Make sure the disk you use for the OE tree supports symbolic links, and you use at least git v1.8.0. And Bitbake doesn't support NFS natively, you need local block storage. |
Install prerequisite packages, as described here: [http://www.openembedded.org/wiki/OEandYourDistro<span style="color:SteelBlue">http://www.openembedded.org/wiki/OEandYourDistro</span>] | Install prerequisite packages, as described here: [http://www.openembedded.org/wiki/OEandYourDistro<span style="color:SteelBlue">http://www.openembedded.org/wiki/OEandYourDistro</span>] | ||
− | For Ubuntu, that's: | + | For Ubuntu 22.04 LTS, that's: |
− | sudo apt-get install sed wget cvs subversion git-core \ | + | sudo apt-get install sed wget cvs subversion git-core git-lfs \ |
coreutils unzip texi2html texinfo docbook-utils \ | coreutils unzip texi2html texinfo docbook-utils \ | ||
− | gawk | + | gawk python3-distutils diffstat help2man make gcc build-essential g++ \ |
desktop-file-utils chrpath default-jre gettext zip libssl-dev | desktop-file-utils chrpath default-jre gettext zip libssl-dev | ||
− | You may need to create a symlink makeinfo->texi2any (in the bin folder) and a symlink makeinfo->/bin/texi2any in the hosttools folder. | + | You may need to create a symlink makeinfo->texi2any (in the bin folder) and a symlink makeinfo->/bin/texi2any in the hosttools folder. |
− | For | + | For EL9, that's: |
− | sudo | + | sudo dnf install gawk make wget tar bzip2 gzip python unzip perl patch \ |
diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \ | diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \ | ||
− | openssl-devel zip python3 perl-Thread-Queue libstdc++-static \ | + | openssl-devel zip python3 perl-Thread-Queue libstdc++-static git-lfs \ |
glibc-devel.i686 libgcc.i686 libstdc++-devel.i686 | glibc-devel.i686 libgcc.i686 libstdc++-devel.i686 | ||
+ | |||
+ | ===Note: Ubuntu 24.04 users=== | ||
+ | A security restriction introduced in an update causes projects to not build from the Ubuntu IDE. This restriction can be disabled system-wide with the following command: | ||
+ | |||
+ | sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 | ||
+ | |||
+ | when booting the PC and building an image, you'll have to do this every boot. | ||
+ | |||
+ | To make the change persistent, add kernel.apparmor_restrict_unprivileged_userns = 0 to /etc/sysctl.conf. | ||
+ | |||
+ | NOTE: This is removing a security restriction which was added to Ubuntu, so do it at your own risk. | ||
+ | |||
+ | |||
+ | |||
+ | ===Setup the build environment:=== | ||
+ | sudo chmod ugo+rw /opt | ||
+ | mkdir -p /opt/openpli | ||
+ | cd /opt/openpli | ||
Clone the openpli repository: | Clone the openpli repository: | ||
git clone https://github.com/OpenPLi/openpli-oe-core.git | git clone https://github.com/OpenPLi/openpli-oe-core.git | ||
− | + | or | |
− | or | ||
− | |||
git clone git@github.com:OpenPLi/openpli-oe-core.git | git clone git@github.com:OpenPLi/openpli-oe-core.git | ||
− | Setup the environment | + | ===Setup the OpenPli environment=== |
cd openpli-oe-core | cd openpli-oe-core | ||
+ | make | ||
− | + | ===Build your first image=== | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | Build your first image | ||
MACHINE=xxXXXX make image | MACHINE=xxXXXX make image | ||
+ | or | ||
+ | cd build | ||
+ | MACHINE=xxXXXX | ||
+ | source env.source | ||
+ | bitbake -k openpli-enigma2-image | ||
To find the valid MACHINE=xxXXXX you can have a look at the .conf file in the meta-folder of the manufacturer MMMMM: | To find the valid MACHINE=xxXXXX you can have a look at the .conf file in the meta-folder of the manufacturer MMMMM: | ||
Line 62: | Line 79: | ||
./openpli-oe-core/build/tmp/deploy/images/xxXXXX/ | ./openpli-oe-core/build/tmp/deploy/images/xxXXXX/ | ||
openpli-homebuild-xxXXXX_usb.zip | openpli-homebuild-xxXXXX_usb.zip | ||
− | |||
===Updating=== | ===Updating=== | ||
Line 71: | Line 87: | ||
Note that by default, the develop branch of Enigma is used when you build an image, assuming that most "homebuild" users will want to build the latest development image. | Note that by default, the develop branch of Enigma is used when you build an image, assuming that most "homebuild" users will want to build the latest development image. | ||
− | If you want to build a release image, you need to | + | If you want to build a homebuild release image, you need to specify the correct branch when building: |
cd openpli-oe-core | cd openpli-oe-core | ||
− | git checkout release- | + | git checkout release-9.0 |
git submodule update --init --recursive | git submodule update --init --recursive | ||
− | ENIGMA2_BRANCH=release- | + | ENIGMA2_BRANCH=release-9.0 MACHINE=xxXXXX make image |
The resulting file will be into: | The resulting file will be into: | ||
Line 82: | Line 98: | ||
openpli-homebuild-xxXXXX_usb.zip | openpli-homebuild-xxXXXX_usb.zip | ||
− | === | + | === EL incompatibilities === |
− | |||
− | |||
− | + | None | |
− | + | === Ubuntu incompatibilities === | |
− | When | + | When running a build on ubuntu, you can run into the issue that the build is aborted by the sanity checker, complaining that "makeinfo" does not exist. Even though the package texinfo is installed. This is caused by the fact makeinfo is now a symlink, which doesn't make it into hosttools. |
− | + | To fix this, execute | |
− | cd /tmp | + | cd <your-build-dir>/build/tmp/hosttools |
− | + | ln -s /usr/bin/texi2any makeinfo | |
− | |||
− | |||
=== Cleaning up the sstate cache === | === Cleaning up the sstate cache === | ||
Line 104: | Line 116: | ||
To address this, make your git directory the current directory, and run | To address this, make your git directory the current directory, and run | ||
./openembedded-core/scripts/sstate-cache-management.sh --cache-dir=./build/sstate-cache --stamps-dir=./build/tmp/stamps --yes | ./openembedded-core/scripts/sstate-cache-management.sh --cache-dir=./build/sstate-cache --stamps-dir=./build/tmp/stamps --yes | ||
− | |||
− | |||
− | |||
− | |||
− | |||
Note that if your cache is old, this may take a long time, as on our buildserver (which does build all images!) the result after a second build run is "1385449 out of 1682374 files will be removed!" | Note that if your cache is old, this may take a long time, as on our buildserver (which does build all images!) the result after a second build run is "1385449 out of 1682374 files will be removed!" | ||
Line 166: | Line 173: | ||
NB: For ports below 1024, like the standard HTTP port 80 in the example above, you need root or administrator privileges. | NB: For ports below 1024, like the standard HTTP port 80 in the example above, you need root or administrator privileges. | ||
+ | |||
+ | |||
+ | =='''Feeds from a USB drive.'''== | ||
+ | Name a folder to /feeds in the root folder of a USB drive. Copy your feeds to that folder. Now insert the USB in your receiver, e.g. mount as /media/usb. | ||
+ | |||
+ | Next, modify the opkg paths (of the .conf files in /etc/opkg). | ||
+ | For example, "all-feed.conf" to file:///media/hdd/feeds/all | ||
+ | |||
+ | Restart and you should be up and running with feeds from the USB. | ||
=='''OpenPLi - Git commands'''== | =='''OpenPLi - Git commands'''== | ||
Here is the link to some basic [[git-commands | <span style="color:SteelBlue">'''git commands'''</span>]]. | Here is the link to some basic [[git-commands | <span style="color:SteelBlue">'''git commands'''</span>]]. |
Revision as of 12:55, 6 September 2024
Contents
'I'm a developer, is there any technical info available from the PLi® team?
Welcome and tell your developer friends about us
Yes and welcome aboard! If you have specific question, you can use our PLi® Third Party Development forum to ask it. You will find we are most accommodating.
This page is about software development using OpenEmbedded-core. If you're looking for the text that used to be on the this page, it's now called developer-information-old.
==
Create your own build
See also this forum post on setting up a development environment: http://openpli.org/forums/topic/18806-openpli-quick-setup-ubuntudebian/
Basically, it boils down to this:
Run Linux. Most of us use either Ubuntu 22.04 or an EL v9 distro, I suggest you do the same, if you don't know what to pick. Stay away from bleeding edge distro's like Fedora, you'll run into all sorts of misery.
Make sure the disk you use for the OE tree supports symbolic links, and you use at least git v1.8.0. And Bitbake doesn't support NFS natively, you need local block storage.
Install prerequisite packages, as described here: http://www.openembedded.org/wiki/OEandYourDistro
For Ubuntu 22.04 LTS, that's:
sudo apt-get install sed wget cvs subversion git-core git-lfs \ coreutils unzip texi2html texinfo docbook-utils \ gawk python3-distutils diffstat help2man make gcc build-essential g++ \ desktop-file-utils chrpath default-jre gettext zip libssl-dev
You may need to create a symlink makeinfo->texi2any (in the bin folder) and a symlink makeinfo->/bin/texi2any in the hosttools folder.
For EL9, that's:
sudo dnf install gawk make wget tar bzip2 gzip python unzip perl patch \ diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \ openssl-devel zip python3 perl-Thread-Queue libstdc++-static git-lfs \ glibc-devel.i686 libgcc.i686 libstdc++-devel.i686
Note: Ubuntu 24.04 users
A security restriction introduced in an update causes projects to not build from the Ubuntu IDE. This restriction can be disabled system-wide with the following command:
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
when booting the PC and building an image, you'll have to do this every boot.
To make the change persistent, add kernel.apparmor_restrict_unprivileged_userns = 0 to /etc/sysctl.conf.
NOTE: This is removing a security restriction which was added to Ubuntu, so do it at your own risk.
Setup the build environment:
sudo chmod ugo+rw /opt mkdir -p /opt/openpli cd /opt/openpli
Clone the openpli repository:
git clone https://github.com/OpenPLi/openpli-oe-core.git or git clone git@github.com:OpenPLi/openpli-oe-core.git
Setup the OpenPli environment
cd openpli-oe-core make
Build your first image
MACHINE=xxXXXX make image or cd build MACHINE=xxXXXX source env.source bitbake -k openpli-enigma2-image
To find the valid MACHINE=xxXXXX you can have a look at the .conf file in the meta-folder of the manufacturer MMMMM:
cd meta-MMMMM/conf/machine ls
If the build is successfull the resulting file will be into:
./openpli-oe-core/build/tmp/deploy/images/xxXXXX/ openpli-homebuild-xxXXXX_usb.zip
Updating
Before every build, update your cloned git repositories with
cd openpli-oe-core make update
Note that by default, the develop branch of Enigma is used when you build an image, assuming that most "homebuild" users will want to build the latest development image.
If you want to build a homebuild release image, you need to specify the correct branch when building:
cd openpli-oe-core git checkout release-9.0 git submodule update --init --recursive ENIGMA2_BRANCH=release-9.0 MACHINE=xxXXXX make image
The resulting file will be into:
./openpli-oe-core/build/tmp/deploy/images/xxXXXX/ openpli-homebuild-xxXXXX_usb.zip
EL incompatibilities
None
Ubuntu incompatibilities
When running a build on ubuntu, you can run into the issue that the build is aborted by the sanity checker, complaining that "makeinfo" does not exist. Even though the package texinfo is installed. This is caused by the fact makeinfo is now a symlink, which doesn't make it into hosttools.
To fix this, execute
cd <your-build-dir>/build/tmp/hosttools ln -s /usr/bin/texi2any makeinfo
Cleaning up the sstate cache
You will notice that after each build, the storage used by the sstate-cache will increase. This is because this cache holds all data required to be able to skip building packages that haven't changed, and previous release data is not purged by bitbake.
To address this, make your git directory the current directory, and run
./openembedded-core/scripts/sstate-cache-management.sh --cache-dir=./build/sstate-cache --stamps-dir=./build/tmp/stamps --yes
Note that if your cache is old, this may take a long time, as on our buildserver (which does build all images!) the result after a second build run is "1385449 out of 1682374 files will be removed!"
Your own feed server
Once you've built your own image, you'd want to keep it up to date. You can just let the box update itself from your build PC using the GUI as if running a full distro. To do that, you first need to build the optional packages that go into the feed:
cd openpli-oe-core MACHINE=xxXXXX make feed
Next, you need to install a webserver of some sort on the build PC, for example Apache2.
As an example, if you've installed apache on Ubuntu, it will share /var/www/html/ with the world. Create a feed subdirectory, and then add a symlink to the "ipk" folder of your build, for example:
sudo mkdir /var/www/html/feeds sudo ln -s ${HOME}/work/openpli-oe-core/build/tmp/deploy/ipk/ /var/www/html/feeds/openpli-homebuild
To tell your box about this feed location, edit your local.conf or site.conf to contain the following lines:
FEED_NAME = "openpli-homebuild" DISTRO_HOST = "mybuildpc.local"
After this, if you run a new image build, it will make the /etc/opkg/ files on the box point to your feed. After installing the built image on your box, you should be able to upgrade with opkg commands or the enigma2 GUI.
No real webserver handy?
If you don't have a real webserver, like Apache or Nginx, handy, there are some other options available to serve files via http.
Using PHP 5.4+:
cd /var/www/html/feeds php -S 0.0.0.0:80
Using Python 2.x:
cd /var/www/html/feeds python -m SimpleHTTPServer 80
Using Python 3.x:
cd /var/www/html/feeds python3 -m http.server 80
Using Ruby:
cd /var/www/html/feeds ruby -rwebrick -e'WEBrick::HTTPServer.new(:Port => 80, :DocumentRoot => Dir.pwd).start'
Using Ruby 1.9.2+:
cd /var/www/html/feeds ruby -run -ehttpd . -p80
Using Perl:
cpan HTTP::Server::Brick # one-time install of perl dependency cd /var/www/html/feeds perl -MHTTP::Server::Brick -e '$s=HTTP::Server::Brick->new(port=>80); $s->mount("/"=>{path=>"."}); $s->start'
Using Busybox HTTPd:
cd /var/www/html/feeds busybox httpd -f -p 80
NB: For ports below 1024, like the standard HTTP port 80 in the example above, you need root or administrator privileges.
Feeds from a USB drive.
Name a folder to /feeds in the root folder of a USB drive. Copy your feeds to that folder. Now insert the USB in your receiver, e.g. mount as /media/usb.
Next, modify the opkg paths (of the .conf files in /etc/opkg). For example, "all-feed.conf" to file:///media/hdd/feeds/all
Restart and you should be up and running with feeds from the USB.
OpenPLi - Git commands
Here is the link to some basic git commands.