From: Alan T. DeKok Date: Thu, 16 Feb 2023 21:31:45 +0000 (-0500) Subject: start of detailed dictionary documentation X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=45ddf6ce86b42d46a630619ae3d2ab2021a632e5;p=thirdparty%2Ffreeradius-server.git start of detailed dictionary documentation --- diff --git a/doc/antora/modules/reference/nav.adoc b/doc/antora/modules/reference/nav.adoc index cf64a2972b2..05acd69fac5 100644 --- a/doc/antora/modules/reference/nav.adoc +++ b/doc/antora/modules/reference/nav.adoc @@ -71,3 +71,22 @@ *** xref:man/radiusd.adoc[radiusd] *** xref:man/radmin.adoc[radmin] *** xref:man/radsniff.adoc[radsniff] + +** xref:dictionary/index.adoc[Dictionaries] +*** xref:dictionary/alias.adoc[ALIAS] +*** xref:dictionary/attribute.adoc[ATTRIBUTE] +*** xref:dictionary/define.adoc[DEFINE] +*** xref:dictionary/enum.adoc[ENUM] +*** xref:dictionary/flags.adoc[FLAGS] +*** xref:dictionary/include.adoc[$INCLUDE] +*** xref:dictionary/member.adoc[MEMBER] +*** xref:dictionary/protocol.adoc[PROTOCOL] +*** xref:dictionary/struct.adoc[STRUCT] +*** xref:dictionary/value.adoc[VALUE] +*** xref:dictionary/vendor.adoc[VENDOR] +*** xref:dictionary/begin-protocol.adoc[BEGIN-PROTOCOL] +*** xref:dictionary/end-protocol.adoc[END-PROTOCOL] +*** xref:dictionary/begin-tlv.adoc[BEGIN-TLV] +*** xref:dictionary/end-tlv.adoc[END-TLV] +*** xref:dictionary/begin-vendor.adoc[BEGIN-VENDOR] +*** xref:dictionary/end-vendor.adoc[END-VENDOR] diff --git a/doc/antora/modules/reference/pages/dictionary/alias.adoc b/doc/antora/modules/reference/pages/dictionary/alias.adoc new file mode 100644 index 00000000000..646a943e332 --- /dev/null +++ b/doc/antora/modules/reference/pages/dictionary/alias.adoc @@ -0,0 +1,98 @@ += The ALIAS keyword + +.Syntax +---- +ALIAS +---- + +.Description +The `ALIAS` keyword a name which references a particular OID. + +:: The name of the attribute. The name can contain alphanumeric +characters, `-`, and `_`. The name should be short and descriptive. ++ +The `ALIAS` names are global, and placed at the dictionary root as +with earlier versions of the server. The `ALIAS` names must not +overlap with the name of any other attribute. + +:: An OID by name `Vendor.Cisco.AVPair`, or by number `26.9.1`. ++ +The full name or number must be given. +.Examples +---- +ALIAS Cisco-AVPair 26.9.1 +---- + +== Purpose of the ALIAS keyword + +The purpose of the `ALIAS` keyword is to allow for easier transition +from ealier versions of FreeRADIUS to version 4. We recognize that +the old attribute names are used in many places such as databases. It +can be difficult to change many thousands of existing entries in a +"live" system. + +As such, we provide some ability to use names which are compatible +with version 3 of FreeRADIUS. + +NOTE: The `ALIAS` names are not enabled by default. They must be +manually enabled on each system by editing the `raddb/dictionary` +file. + +== v3 Compatible names + +All of the attributes have been renamed from v3. This change was +necessary in order to support new funtionality in v4. The +unfortunate side effect of this change is that all of the names in +SQL, LDAP, and the "files" module are incompatible with v4. + +We recognize that is is difficult to change every entry in a +database, especially when there's no clear mapping between the +"old" and "new" names. This renaming is made more complex because +the "new" names need to be grouped and arranged in ways that the +old ones were not. + +The "old" names were all in flat lists, so that` User-Name` appeared +next to `Cisco-AVPAir`. This organization was simple enough to work +for 20 years, but its time has come. The new names are +hierarchical, so that the organization is nested by definition. + +For v4, the `Cisco-AVPair` attribute is called `AVPair`, and it lives +inside of the `Cisco` namespace, which in turn lives inside of the +`Vendor-Specific` namespace. So the new name for `Cisco-AVPair` is +`Vendor-Specific.Cisco.AVPair`. + +This process continues for many thousands of vendor-specific +attributes. + +Happily, it is possible to (mostly) use the old names with v4. +There are limitations, but it will mostly work. The main reason +for enabling the old names is to try out v4 with a database that is +also used by v3. This lets you test that v4 works, without going +through a complex "upgrade everything" process. + +The old v3 names are in "alias" dictionaries, in the `${dictdir}` +directory. To find out where this directory is on your local +system, run `radiusd -h` or `radclient -h`. Then look for the `-D` +command-line option, and it will tell you where the dictionary +files are located. + +The v3 names are in `${dictdir}/radius/alias/VENDOR.txt` where +`VENDOR` is the (lowercase) name of the vendor, which is taken from the `VENDOR` +definition in the v3 dictionaries. + +You will need to add a `$INCLUDE` line for each vendor-specific +dictionary which is used by your local system. The default v4 +dictionaries do not enable all of v3 compatibilty names. + +Yes, we recognize that this process is a bit of work. However, we +wish to encourage everyone using v4 to upgrade to using the new v4 +features. Our experience shows that if we automatically enable +"compatibility functions", then those compatiblity functions will +be used for a decade. So we need to find a balance between +upgrades and ongoing support. Easy upgrades will mean complex +ongoing support. Complex upgrades make ongoing support easier, but +also make it less likely that people will upgrade. + + +// Copyright (C) 2023 Network RADIUS SAS. Licenced under CC-by-NC 4.0. +// Development of this documentation was sponsored by Network RADIUS SAS. diff --git a/doc/antora/modules/reference/pages/dictionary/attribute.adoc b/doc/antora/modules/reference/pages/dictionary/attribute.adoc new file mode 100644 index 00000000000..fbd97aac2b0 --- /dev/null +++ b/doc/antora/modules/reference/pages/dictionary/attribute.adoc @@ -0,0 +1,84 @@ += The ATTRIBUTE keyword + +.Syntax +---- +ATTRIBUTE [] +---- + +.Description +The `ATTRIBUTE` keyword defines a name, number, and data type mapping. + +:: The name of the attribute. The name can contain alphanumeric +characters, `-`, and `_`. The name should be short and descriptive. ++ +As the names are hierarchical, the name is scoped to its parent. So +the name `Counter` can mean different things, depending on its +context. + +:: The number of the attribute. Numbers can be specified as +decimal (`19`), or as hex (`0xffee`). ++ +Numbers can also be specified as object identifiers (OIDs), `26.1.9`, +or as partial OIDs (`.14`). ++ +There are a number of complex rules and interpretations of OID values +which are difficult to clearly document using simple examples. This +complex behavior is only needed for rare situations. In short, if you +are creating complex protocol dictionaries, you need to understand +those protocols before creating the dictionaries. On the other hand, +if you are creating simple protocol dictionaries, there is no need to +understand any complex rules around the use of OIDs. + +:: A xref:type/index.adoc[data type] ++ +A few data types can be marked up as being fixed size, by adding an +"array-style" suffix, such as with `octets[3]`. See below for details. + +:: The flag field is optional, and is usually protocol-specific. See +the xref:dictionary/flags.adoc[FLAGS] page for definitions of common +flags. ++ +Please see the protocol dictionaries for examples of these flags. +Only the flags which are used in a protocol dictionary are valid, +and those flags are only valid for that protocol dictionary. ++ +The old-style RADIUS syntax of specifying a vendor name in the `flags` +field is not supported. + +Common flags and meanings +[options="header"] +[cols="30%,70%"] +|===== +| Name | Description +| `array` | For fixed-size types, declare that the contents of the packet can have an array of this value. +| `clone=...` | For `tlv` types, clone (or copy) child definitions from another `tlv` type +| `internal` | This attribute is internal to the server, and will never be sent "on the wire" +| `ref=...` | For `group` types, specify which attributes are allowed to be in this `group` +|===== + + +.Examples +---- +ATTRIBUTE Foo 1 string[3] +ATTRIBUTE Bar 2 octets +ATTRIBUTE baz 3 ipv4addr +---- + +== Fixed Size Data types + +The following data types can be marked up as having fixed size, by +using an array suffix, e.g. `octets[14]`. + +Fixed Size types and meanings +[options="header"] +[cols="30%,70%"] +|===== +| Type | Description +| `octets[n]` | Declare that this attribute uses `n` bytes of `octets` data +| `string[n]` | Declare that this attribute uses `n` bytes of `string` data +| `struct[n]` | Declare that this structure uses `n` bytes for itself, including all child xref:dictionary/member.adoc[MEMBER]s +|===== + + +// Copyright (C) 2023 Network RADIUS SAS. Licenced under CC-by-NC 4.0. +// Development of this documentation was sponsored by Network RADIUS SAS. diff --git a/doc/antora/modules/reference/pages/dictionary/define.adoc b/doc/antora/modules/reference/pages/dictionary/define.adoc new file mode 100644 index 00000000000..20ca48f134f --- /dev/null +++ b/doc/antora/modules/reference/pages/dictionary/define.adoc @@ -0,0 +1,46 @@ += The DEFINE keyword + +.Syntax +---- +DEFINE [] +---- + +.Description + +The `DEFINE` keyword defines a name and data type mapping for +attributes which do not have numbers associated with them. These +attributes are generally not sent in any protocol, or the underlying +protocol is string-based, and not binary-packed data. + +:: The name of the attribute. The name can contain alphanumeric +characters, `-`, and `_`. The name should be short and descriptive. ++ +The name of this `DEFINE` is defined only within the context of its +parent attribute. + +:: A xref:type/index.adoc[data type]. + +:: The flag field is optional, and is usually protocol-specific. See +the xref:dictionary/flags.adoc[FLAGS] page for definitions of common +flags. ++ +Please see the protocol dictionaries for examples of these flags. +Only the flags which are used in a protocol dictionary are valid, +and those flags are only valid for that protocol dictionary. + +Common flags and meanings +[options="header"] +[cols="30%,70%"] +|===== +| Name | Description +| `clone=...` | For `tlv` types, clone (or copy) child definitions from another `tlv` type +|===== + +.Examples +---- +DEFINE Foo string +DEFINE Bar ipv4addr +---- + +// Copyright (C) 2023 Network RADIUS SAS. Licenced under CC-by-NC 4.0. +// Development of this documentation was sponsored by Network RADIUS SAS. diff --git a/doc/antora/modules/reference/pages/dictionary/include.adoc b/doc/antora/modules/reference/pages/dictionary/include.adoc new file mode 100644 index 00000000000..88ba03e74ed --- /dev/null +++ b/doc/antora/modules/reference/pages/dictionary/include.adoc @@ -0,0 +1,28 @@ += The $INCLUDE keyword + +.Syntax +---- +$INCLUDE +$INCLUDE- +---- + +.Description +The `$INCLUDE` keyword includes another dictionary from `` into the dictionary. The file must exist, and be readable by the server. + +The `$INCLUDE-` acts above, except it does nothing if the file does not exist. If the file exists and is not readable by the server, an error is produced. + +:: The file name on the file system. ++ +Relative filenames such as `dictionary.foo` are loaded from the same directory as the current dictionary file. ++ +Absolute filenames such as `/usr/share/freeradius/dictionary.foo` are loaded starting at the root of the file system. + +NOTE: Attribute definitions cannot span multiple files. This include `BEGIN-` and `END-` blocks, which must all be contained in the same file. + +.Example +---- +$INCLUDE dictionary.foo +---- + +// Copyright (C) 2023 Network RADIUS SAS. Licenced under CC-by-NC 4.0. +// Development of this documentation was sponsored by Network RADIUS SAS. diff --git a/doc/antora/modules/reference/pages/dictionary/index.adoc b/doc/antora/modules/reference/pages/dictionary/index.adoc new file mode 100644 index 00000000000..8d1d0eebd6f --- /dev/null +++ b/doc/antora/modules/reference/pages/dictionary/index.adoc @@ -0,0 +1,152 @@ += Dictionaries + +The `dictionary` files define names, numbers, and +xref:type/index.adoc[data types] for use in the server. In general, +the dictionary files are defined by industry standard specifications, +or by a vendor for their own equipment. The dictionary files in the +`share` directory _should not be edited_. Editing those files will +likely break the server. + +Local dictionary definitions should be placed in the +`raddb/dictionary` file. + +== Concepts and Background + +The dictionaries are necessary because many networking protocols do +not send humanly-readable names over the network. Instead, the +protocols send sequences of bytes to represent concepts, where each +byte has a particular meaning. + +At the same time, it is useful for administrators to write policies +based on names, and not on numbers. No one wants to write a policy of +the form `if concept 15 has value "foo", then do ...`. + +The dictionaries solve this problem by enabling the server to decode +the magic numbers into humanly-readable names. The administrator can +then write policies of the form `if the User-Name has value "bob", +then do...`. + +These policies are much simpler to create and understand. + +NOTE: The use of "json over HTTP" for almost everything is a few decades +newer than RADIUS, DHCP, etc. Perhaps one day everyone will move to +that. Until then, we are stuck with the dictionaries. + +The dictionaries also enable the server to decode protocols using +xref:type/index.adoc[data types], and then present that decoded data +to the administrator. As with the other examples above, it would be +very difficult write policies based on "raw hex" data: `if User-Name is 0x626f62 ...`. + +The dictionaries solve the data type problem by associating +xref:type/index.adoc[data types] with a name and number. That way the +bytes `0x7f000001` can be presented to the administrator as an IPv4 +address of `127.0.0.1`. + +This association is two-way, protocols can get decoded to +understandable names and data types, and those names and data can get +encoded in protocols. + +=== Dictionaries are always local + +In almost all cases, the names defined in a dictionary have no meaning +outside of the server. The names are never sent "on the wire" between +server and client. Editing the dictionary names on one system will +_not change_ the names used by another system. + +The names are also local to each implementation. FreeRADIUS has +chosen a set of names for itself, which are based on specifications +and on vendor definitions. In nearly all cases, these names are +either the same as the external definition, or are very similar to the +external definition. + +In general, the only reason for us to change the names is that the +names conflict with another name. It is not possible to have the same +name mean two entirely different things. + +Vendors can submit new dictionaries to us via +mailto:dictionary@freeradius.org[email], or via +https://github.com/FreeRADIUS/freeradius-server/[GitHub]. + +=== Names are Case-insensitive + +When names are printed, they are printed as the name given in the +dictionaries. When a name is looked up in the dictionaries, the +lookup is done in a case-insensitive manner. + +=== Names are Hierarchical + +In earlier versions of FreeRADIUS, the names were global. The global +names for attributes caused issue with implementations, as noted in +https://www.rfc-editor.org/rfc/rfc6929.html#section-2.7.1[RFC 6929 +Section 2.7.1]. This limitation caused the names to be long, and have +"vendor prefixes", such as with `Cisco-AVPair`. + +The names in FreeRADIUS are now hierarchical. In most cases, the old +names are simply split into separate sub-names with a period (`.`). +For example, the previously mentioned `Cisco-AVPair` now becomes +`Cisco.AVPair.`, or `Vendor-Specific.Cisco.AVPair`. + +If there is still a need to use the old names, the +xref:dictionary/alias.adoc[ALIAS] keyword can help. See +`raddb/dictionary` for additional documentation. + +== Files and Location + +The dictionaries are placed into a `share` directory, which is usually +in a location such as `/usr/share/freeradius/`. The definitions for +each individual specification or vendor dictionary are placed in files +with the appropriate name. For example, `dictionary.rfc2865` or +`dictionary.cisco`. + +We reiterate that these dictionaries should not be edited. These +dictionaries ship with the server, and any new installation (or +upgrade) will over-write local changes. + +Local edits should be kept to the `raddb/dictionary` file, which will +not be overwritten on an upgrade. + +== File Syntax + +The dictionary file format follows the standard RADIUS dictionary +syntax. In many respects, the format has had only minor changes since +the original Livingston RADIUS server in 1993. + +The file format is simple, and line-oriented. Blank lines are +ignored. Hash (`#`) characters are comments, and cause everything +after the hash character to be ignored, up to the end of the line. + +Every non-blank line starts with a keyword, as given in the table +below. In most cases, the main keywords of interest are +xref:dictionary/attribute.adoc[ATTRIBUTE] and xref:dictionary/value.adoc[VALUE]. + +.Dictionary Definition Keywords +[options="header"] +[cols="30%,70%"] +|===== +| Keyword | Description +| xref:dictionary/alias.adoc[ALIAS] | Define a name which references an `ATTRIBUTE` +| xref:dictionary/attribute.adoc[ATTRIBUTE] | Define a name, number, and data type mapping +| xref:dictionary/define.adoc[DEFINE] | Define a name and data type mapping +| xref:dictionary/enum.adoc[ENUM] | Define a named enumeration of values for use with multiple `ATTRIBUTE`s +| xref:dictionary/flags.adoc[FLAGS] | Set flags for subsequent definitions +| xref:dictionary/include.adoc[$INCLUDE] | Include another dictionary file +| xref:dictionary/member.adoc[MEMBER] | Define a member of a `STRUCT` +| xref:dictionary/protocol.adoc[PROTOCOL] | Define a protocol like `RADIUS` or `DHCPv4` +| xref:dictionary/struct.adoc[STRUCT] | Define a structure which can contain `MEMBER`s +| xref:dictionary/value.adoc[VALUE] | Define a name for a particular value of an `ATTRIBUTE` +| xref:dictionary/vendor.adoc[VENDOR] | Define a name and number for a vendor +|===== + +The following keywords still XXX + +.Dictionary Nesting Keywords +[options="header"] +[cols="30%,70%"] +|===== +| xref:dictionary/begin-protocol.adoc[BEGIN-PROTOCOL] | Begin defining a protocol dictionary +| xref:dictionary/end-protocol.adoc[END-PROTOCOL] | End a protocol dictionary +| xref:dictionary/begin-tlv.adoc[BEGIN-TLV] | Begin defining children of a `tlv` data type +| xref:dictionary/end-tlv.adoc[END-TLV] | End defining children of a `tlv` data type +| xref:dictionary/begin-vendor.adoc[BEGIN-VENDOR] | Begin defining vendor-specific attributes +| xref:dictionary/end-vendor.adoc[END-VENDOR] | End defining vendor-specific attributes +|===== diff --git a/doc/antora/modules/reference/pages/dictionary/member.adoc b/doc/antora/modules/reference/pages/dictionary/member.adoc new file mode 100644 index 00000000000..bd202cde419 --- /dev/null +++ b/doc/antora/modules/reference/pages/dictionary/member.adoc @@ -0,0 +1,71 @@ += The MEMBER keyword + +.Syntax +---- +MEMBER [] +---- + +.Description + +The `MEMBER` keyword defines a name and data type mapping for members +of a parent xref:dictionary/struct.adoc[STRUCT] definition. The definitions +preceding a `MEMBER` must be either another `MEMBER`, or be a `STRUCT`. + +:: The name of the attribute. The name can contain alphanumeric +characters, `-`, and `_`. The name should be short and descriptive. ++ +The name of this `MEMBER` is defined only within the context of its parent `STRUCT`. + +:: A xref:type/index.adoc[data type], or the special type `bit`. ++ +A few data types can be marked up as being fixed size, by adding an +"array-style" suffix, such as with `bit[3]`. See below for details. + +NOTE: Only `MEMBER` definitions can contain `bit` fields. While `bit` +fields do not have to start and end on byte boundaries, changing from +a non-`bit` type to a `bit` type (and vice-versa) can only be done on +a byte boundary. + +:: The flag field is optional, and is usually protocol-specific. See +the xref:dictionary/flags.adoc[FLAGS] page for definitions of common +flags. ++ +Please see the protocol dictionaries for examples of these flags. +Only the flags which are used in a protocol dictionary are valid, +and those flags are only valid for that protocol dictionary. + +Common flags and meanings +[options="header"] +[cols="30%,70%"] +|===== +| Name | Description +| `array` | For fixed-size types, declare that the contents of the packet can have an array of this value. +| `clone=...` | For `tlv` types, clone (or copy) child definitions from another `tlv` type +|===== + +.Examples +---- +STRUCT Foo +MEMBER Bar bit[3] +MEMBER Baz bit[5] +MEMBER Name string +---- + +== Fixed Size Data types + +The following data types can be marked up as having fixed size, by +using an array suffix, e.g. `octets[14]`. + +Fixed Size types and meanings +[options="header"] +[cols="30%,70%"] +|===== +| Keyword | Description +| `bit[n]` | Declare that this attribute uses `n` bits, treated as an unsigned integer +| `octets[n]` | Declare that this attribute uses `n` bytes of `octets` data +| `string[n]` | Declare that this attribute uses `n` bytes of `string` data +|===== + + +// Copyright (C) 2023 Network RADIUS SAS. Licenced under CC-by-NC 4.0. +// Development of this documentation was sponsored by Network RADIUS SAS. diff --git a/doc/antora/modules/reference/pages/index.adoc b/doc/antora/modules/reference/pages/index.adoc index 2d57e729035..87324137254 100644 --- a/doc/antora/modules/reference/pages/index.adoc +++ b/doc/antora/modules/reference/pages/index.adoc @@ -2,6 +2,7 @@ We have a lot of stuff +* xref:dictionary/index.adoc[Dictionary] where attribute names and data types are defined * xref:type/index.adoc[Data Types] in the server * xref:unlang/index.adoc[Unlang] syntax * xref:xlat/index.adoc[Dynamic expansions] i.e. "xlat"s.