Packets are the most fundamental part of MBPacketManager, and what makes the application do anything meaningfull.
This document describes how packets are created, the different kinds of packet on the system, and the meaning of MBPM_PacketInfo file that describes the packet, aswell as the design philosophy behind packets and the problems they are intended to solve.
1 Design philosophy
Packets are designed to modularize functionality in some way, and to make it easy to transfer between computers, aswell as keeping track of exactly what this functionality depends on.
MBPacketManager initially started as purely a way to separate C++ projects into multiple libraries, but has since then evolved to support generic packets.
In order to do this so is there a separation from the dependancy and package management from the semantics of the package installed.
But one of the main goals of this project, more of which can be found
here, is to also make it easy to create packets yourself and use locally. Packets shouldn't be a hassle to create and distribute and utilize, that would remove half of the reason to use them in the first place. This means that the structure of the packet is extremly free and few things are mandated, which also ties in with the previous point in that package managment should work for every type of packet.
1.1 Complete portability
mbpm isn't just designed to be a way to distribute packages between computer, but more importantly a way use the functionality provided for every computer. This means that packets should be designed to be completely portable, with semantics that makes the usable for every kind of computer.
This for example means that it's not enough to just be able to distribute a C++ packet in a compiled form, but that the packet has to be compileable for an arbitrary computer.
1.2 No arbitrary code execution
One of the most important principles, and one of the principles that make the philosophy of mbpm different form other kind of packet managers, is that the installation/usage of a packet MUST NOT use a user provied script in order to be successfully installed.
Another purpose of packet managers is to modularize and make the dependancies of projects more managable, but if the installation of a packet that uses arbitrary scripts ever go wrong, so is the ability to understand and fix the problem is severely limited by needing to reverse engineer the installation process.
It also makes the invariants and assumption made by mbpm harder to enforce, and it also enforces implicit dependancies on the computer in order to able to execute the scripts in question.
The installation of a packet should be handled solely by an appropriate
extension in order to make the behaviour well documented and reproducible, and to ensure that errors in the build process are either easily identified, or centraly fixable.
1.3 Complete track of exported files
Some packet managers doesn't support true removal of a packet, because these packet might create files that are outside of the location of the directories managed by the packet manager, or because few restrictions are placed on it's installation script.
Packets used by mbpm should always have complete knowledge about the files they use on the system, so that removal of the packet doesn't result in broken symlinks or superflous directories not being removed.
2 Packet structure and creating a packet
In order to create a packet so is exactly one thing needed: A valid MBPM_PacketInfo file, it's content is
described further down.
Every directory which contains a MBPM_PacketInfo is interpreted as a packet directory for the purpose of
determining user packet, and for determining the
installed packets.
The content of a packet is simply a directory structure, all rooted under to / directory of the packet, the directory containing the MBPM_PacketInfo. This means that all packets contains atleast the file "/MBPM_PacketInfo". While some different kind of packet type may have different restrictions to how packet can be composed, so is it not something that is enforced during the distribution, uploading and downloading, of packets.
While there only is one file that is required to exist, so are there a couple of files and directories that are assumed not to.
All files/directories with a name begining with "MBPM_" can be created, removed, and modified by MBPacketManager, aswell as the special directory ".mbpm". Having files in these directories as a part of the packet should therefore be seen as undefined behaviour.
There also exist a small set of special directories with semantics intended with common semantics intended to be used by multiple packet types.
2.1 MBPM_Builds
This directory can optionally contain the compiled version of some packet. For C/C++ projects so might this contain the compiled executable or static library, and for documentation so can this be the finished pdf etc. The exact semantics depends on the packet type.
2.1.1 C/C++
The directory contains a subdirectory for each build configuration, which in turn contains the compiled version for a specific configuration. The special directory "default" can be used to supply a build that isn't specific to any compile configuration.
Exporting mbpm looks in the
default export configuration for the
exported executables/library, or in the default directory if the default export configuration is not present.
This means that the default directory can be used to reliable distribute precompiled versions of a packet. However, it should be noted that following the MBBuild standard for compilation is highly encouraged.
2.2 MBPM_BuildFiles
This directory contains the build files needed for building a specific packet, and it's content like MBPM_Builds is highly dependant on the packet type. This is it's use for the current builtin supported packet types:
2.2.1 C/C++
The build files for each configuration is stored in "MBPM_BuildFiles/{ConfigurationName}", which includes the objects produced during compilation, and MBBuildDependancyInfo, a file listing the dependancies for each object and it's last compilation time, used for determining which files needs to be compiled and executables relinked for succesive compilations.
3 MBPM_PacketInfo
This is the most central configuration file in the project, the file defining a packet. This configuration is a regular JSON file, the content must be single JSON object containing only field describe here. There are a couple of fields that are mandatory, which must be included, and omitting them is treated as an error when parsing the file. Optional fields all have well defined standard values.
3.1 Mandatory fields
3.1.1 PacketName
This field MUST be a string, describing the name of the packet. This name should be unique in order to avoid clashes, but there are ultimately no safegaurds against name's clashing. This field being an empty string is also treated as an error.
3.1.2 Dependancies
This field MUST be a list of strings, which are the names of the packets this packet depend on. These dependancies are used for example when installing a packet, and determining it's
packet depth.
3.2 Optional fields
3.2.1 Type
Packet type is one of the extension mechanisms for mbpm. While the commands install,upload,update,get should all work for every packet type in the same ways so are there commands which might differ, compile and export being the prime examples. Types might also be used by
extensions to override default behaviour.
The only commands which have different semantics depending on the packet are compile and export. Both commands need to examine the type of packet in order to determine what to do, compilation being the command used for creating a executable or similar, while export is tasked with making the result usable, both operations naturally require specific information about the packet.
The default value for Type is "C++".
3.2.2 TypeInfo
A JSON object which is used pass type specific info to MBPM, and it's semantics is therefore dependant on the packet.
No matter the type of packet so should an empty TypeInfo always be a valid value, and the default is also the empty json object.
Here the TypeData objects for the builtin types listed:
3.2.3 C/C++
A json object which might optionally contain the following fields
3.2.4 Fields
3.2.5 Attributes
This field field MUST be a list of strings.
Attributes are a way to add properties to a packet which can either be interpreted directly by mbpm to modify how the packet is treated, or as a way for the user to group packets and assign their own semantic. They can for example be used in a
packet specification so sort for packets with specific attributes. The default value for the attributes field is an empty list.
The default behaviour for mbpm for an attribute is to ignore it. Attributes are also allowed to be given special meaning by extensions, and may differ for different packet types.
Here the attributes which are builtin recognised and acted upon by mbpm listed:
3.2.5.1 NonMBBuild
Emits a cringe warning when installing the packet.
4 Location types
Another concept used by mbpm is packet location. mbpm is designed to be able to develop packages locally and replace the installed packet with these, and to execute comands in a large amount of packets simultaneously. But this means that there is a distinction among packet depending on where on the system they are located, which is refered to as the location type. This location type does however have no effect on the semantics or contents of the packet.
There are currently 4 different location types defined by mbpm, all of which can be used in a
packet specification in order to select packets to execute commands on.
How these packets are searched for and determined is highly dependant on the
installation, and to get a complete picture of the filesystem persistance so is a look at page recommended. How these things relate to packets are also described here however.
4.1 Installed packet
A installed packet is every packet which is under the "MBPM_PACKETS_INSTALL_DIRECTORY" environemnt variable.
"MBPM_PACKETS_INSTALL_DIRECTORY" must point to a directory on the filesystem, and it is in this directory that packets are placed when installed by mbpm, and the directory name is the same as the packet name. When determining wheter or not a installed packet with a given name on the system exists, so is the first step to determine wheter or not a directory with this packet name exists under MBPM_PACKETS_INSTALL_DIRECTORY, and if it does, determine if it has a valid MBPM_PacketInfo file with the same packet name. Any of these steps being false means that the packet is not found, and considering as not existing on the system.
Installed packets have the semantics as being the packets all other packets on the system depend on. A user packet migh be developed separately from the installed packet, but that user packet in turn depend only on installed packet. Exactly what this means naturally depends on the packet type, for "C/C++" so is this related to how the packets are built, all packets are linked only to the installed packets.
4.2 User packet
The file $MBPM_PACKETS_INSTALL_DIRECTORY/UserDirectories.txt contains a list of directires separated by a new line. Whenever a user packet with a particular name is searched for, so is each every sub directory of every directory in UserDirectories.txt examined, but not recursivly. If a subdirectory contains a valid MBPM_PacketInfo file, with the packet name searched for so is a match found, and otherwise it fails.
4.3 Local packet
A local packet is a packet which is not determined to be either a user packet, or a installed packet. This location type is used during
packet specification when a path to a packet is specified instead of a name.
4.4 Remote packet
A remote packet is a packet that is assumed to exist at a
remote location. They are identified by a name only, and the location and protocol used to access this is determined by the
remote configuration.