X11 | 8bit characters | xmodmap , latin1 keysyms , compose sequences | app resources, xterm | xmond | xev | mem footprints
Various | Home

Using eight-bit-characters in X11.

Various methods for generating special characters in X11, some background and about the requirements.

Please note, that this page doesn't cover the XKEYBOARD extension.

(If you are using the text console or a video text terminal: This is not covered here. Their mechanisms for configuring/typing special characters are very system specific. Numerous modern Unix flavours support this for their console [setkeys(1)/loadkeys(1)/dumpkeys(1), etc]. Video text terminals might have a special built-in "compose" mechanism, probably to be activated by a built-in menu system in advance.)

The following doesn't concern only 8bit "special characters" but all those characters which you can't find literally on your caps. On some national keyboards even some important ASCII characters are missing, e.g., {, [, ], }, @, \, |, ~.

This page contains latin1 (western europe 8-bit) characters for illustration. Your browser/font might not display them.

(By the way, using the Euro Symbol is on-topic, in Europe. Then you need latin9.)

A good explanation of the X11 keyboard model is in the xkeycaps(1) manual. (Probably, for now, there's enough information in here, to at least start without having read it.)

If you use the XKB-extension (in contrast to me, because i am using an X11R5 at the time of this writing), i.e. if xdpyinfo(1) shows "XKEYBOARD" in the extension section, and if you try to solve special problems with it, mainly getting it running at all, then the following might not be useful. But still see more documentation and system specific problems for some hints about configuring XFree86 (where problems with XKB usually occur). Nevertheless, the plain xmodmap way described in here always works (although it might mean some work in the worst case).

Content:

Before you start with keymapping,

There are at least five ways to type special characters:

And if you have problems:

Necessary requirements.

A List of both the literal characters and their symbolic KeySym-names in X11 might be very handy.

But before you start looking at the keymapping in X11, confirm the following:

1. You have your special characters literally on your keyboard caps

If you have a Keyboard with eight-bit characters on it (ä, ö, ü, ø, æ, etc), then inspect your keymapping with xev(1) (follow this link if you don't have it; or alternatively look for xkeycaps(1)). Press the caps in question, it should look like the following. Only the bold parts are important: 

	KeyPress event, serial 15, synthetic NO, window 0x4400001,
	root 0x29, subw 0x0, time 1444453669, (112,112), root:(121,362),
	state 0x00, keycode ... (keysym 0xe4, adiaeresis), same_screen YES,
	XLookupString gives 1 characters:  "ä"

Note, that the symbolic name (the KeySym name) usually is a long term identifier, and not the character itself (the only exceptions are the alphanumerical characters from the plain ASCII alphabet). So, in this example, the KeySym name is adiaeresis instead of ä (a-umlaut).

2. Using the Mode_switch/AltGraph mechanism

The most convenient method to get characters which are not literally on your keyboard, is simultaneously pressing an AltGraph key together with some other key. This mechanism is very similar to getting a different character while pressing Shift. (If you don't have a cap labeled AltGr, you can chose another free one as well.)

There are three requirements to make AltGr work:

As an illustration:

(By the way, the '0x14' in parentheses behind Mode_switch is just the hexadecimal value of my Keycode for this cap.)

I arbitrarily associated Mode_switch with "mod3", but you can use any modifier (from mod1 to mod5), which is still free (i.e. not mapped with anything different already). Well, in practice, you shouldn't use mod1, as some programs wrongly (hardcoded) expect this one to be associated with Meta.

See some hints on xmodmap file for more background, and what programs to use to make all this easier, if you still have problems.

(BTW, there's a detailed instruction from Helmut Richter at http://www.lrz-muenchen.de/services/software/x11/xmodmap_en/ for making heavy use of Mode_switch for ISO8859-1.)

Now, if you hold AltGraph and press 'a', while xev(1) is running, it should look like the following, only the emphasized parts are important: 

    KeyPress event, serial 15, synthetic NO, window 0x5000001,
    root 0x29, subw 0x0, time 2948632304, (126,89), root:(344,392),
    state 0x0, keycode 20 (keysym 0xff7e, Mode_switch), same_screen YES,
    XLookupString gives 0 characters:  ""

    KeyPress event, serial 15, synthetic NO, window 0x5000001,
    root 0x29, subw 0x0, time 2948633524, (126,89), root:(344,392),
    state 0x20, keycode 84 (keysym 0xe4, adiaeresis), same_screen YES,
    XLookupString gives 1 characters:  "ä"

(again: the keycode will certainly vary if you don't use the same keyboard like me)

You don't have to verify the following - it's just for your interest:
'0x20' (from the state field) is hexadecimal and means '32'. If you enumerate the logical modifiers above [shift, lock, control, mod1, mod2, ...] with powers of 2 [1, 2, 4, 8, etc.] you'll see that 'mod3' is decimal 32, hexadecimal 20. In X11, the logical modifier settings are just stored as a bit-field internally. If you have mapped it onto another modifier from Mod1-Mod5, then this value will be according.

(One could also patch xev to report modifiers in a human readable way - but it's not really useful, just handy and convenient. xkeycaps(1) will do it out of the box, and xmodmap(1) tells you the mapping of all in one.)

(For two XFree86-3 specialities: Danish-HOWTO, "Getting the AltGr key to work under X11")

3. Using the compose mechanism

Another method to insert eight bit characters is the 'Compose Mechanism'.
On some keyboards, you'll find an accordingly labeled key. Otherwise, you can map any available cap. By the way: The XKB extension alternatively allows to activate this with a key combination (e.g. Shift-AltGr or Shift-RightCtrl).

The KeySym name, which you'll need for the mapping, is Multi_key (hint: you press several keys in a row).
An example is hitting Multi_key, afterwards 'O' and finally '/' [slash], which results in 'Ø' [Oslash].

Originally (X11R5) , this mechanism works "Xserver-internally" and thus you don't need to map it as a logical modifier.

Basically, you only need to map a free cap: 

    $ xmodmap -e 'keycode ... = Multi_key'
...or if you have none: map a combination, i.e. adding it to Mode_switch (or Control_R): 
    $ xmodmap -e 'keycode ... = Mode_switch Multi_key'

If the compose-mechanism is not working for you:

Because of the last point, it's not really possible to debug this with xev(1), as it doesn't support the compose mechanism. (But you certainly can confirm, that "Multi_key" is properly configured!)

If it works for you, see a list of possible combinations of keys and their 'compose'-result. They are rather easy to remember, but you have to type a lot.

4. Using the dead key mechanism

This is another, automatic way of composing. dead means that the actual generation of the character is delayed until you have pressed a second cap: until then the key seems "dead".

This method allows you to compose every (allowed) accented character by just typing the according dead_accent first and the according base character afterwards.

As example: <dead_grave> and then <a> results in 'à'. Unfortunately there are different implementations among X11R5 vs. R6 and MIT vs XFree86. See also the following pages (not always Linux specific at all).

Just as an example: various dead_characters from MIT(/Solaris) X11R6 with a possible resulting latin1 character. 

   dead_grave      + a = à
   dead_acute      + a = á
   dead_circumflex + a = â
   dead_cedilla    + c = ç
   dead_tilde      + n = ñ
   dead_diaeresis  + o = ö
Note: On Solaris2.5[.1] the according KeySym-names (as Sun extension) are not included in /usr/openwin/include/X11/keysymdef.h but - with different names - in /usr/openwin/lib/X11/XKeysymDB: 
SunFA_Grave
SunFA_Acute
SunFA_Circum
SunFA_Cedilla
SunFA_Tilde
SunFA_Diaeresis

[5. Overloading the keymapping in certain applications]

This is not directly related to X11 because it's application specific. It is convenient for example, if you mainly work with xterms and a webbrowser. Two prominent examples to map special charactes:

[6. Cut&Paste the wanted characters

Both quite easy and unhandy.

[7. Using the Meta key (in some applications). Unusable]

For example in xterm(1), you can set the 8th bit of each character if you also press "Meta". But the resulting characters are in no memorable way related to the original 7bit character. It's definitely not usable for all day work. 
ASCII:
    |   | ! | " | # | $ | % | & | ' |
    | ( | ) | * | + | , | - | . | / |
    | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
    | 8 | 9 | : | ; | < | = | > | ? |
    | @ | A | B | C | D | E | F | G |
    | H | I | J | K | L | M | N | O |
    | P | Q | R | S | T | U | V | W |
    | X | Y | Z | [ | \ | ] | ^ | _ |
    | ` | a | b | c | d | e | f | g |
    | h | i | j | k | l | m | n | o |
    | p | q | r | s | t | u | v | w |
    | x | y | z | { | | | } | ~ |del|

(literal ISO8859-1 characters following, your browser might not display them)

With "Meta":
    |   | ¡ | ¢ | £ | ¤ | ¥ | ¦ | § |
    | ¨ | © | ª | « | ¬ | ­ | ® | ¯ |
    | ° | ± | ² | ³ | ´ | µ | ¶ | · |
    | ¸ | ¹ | º | » | ¼ | ½ | ¾ | ¿ |
    | À | Á | Â | Ã | Ä | Å | Æ | Ç |
    | È | É | Ê | Ë | Ì | Í | Î | Ï |
    | Ð | Ñ | Ò | Ó | Ô | Õ | Ö | × |
    | Ø | Ù | Ú | Û | Ü | Ý | Þ | ß |
    | à | á | â | ã | ä | å | æ | ç |
    | è | é | ê | ë | ì | í | î | ï |
    | ð | ñ | ò | ó | ô | õ | ö | ÷ |
    | ø | ù | ú | û | ü | ý | þ | ÿ |

Both tables originate from Solaris, /usr/pub/iso.

To switch off this behaviour in xterm(1) (because it's the default), you need to have "XTerm.eightBitInput: False" in your X11 Resources. (Don't worry, it doesn't affect typing 8-bit charactes with the other, correct mechanisms.)

Then, xterm(1) converts <Meta-key> to <ESC><key>, which is also useful, if you use emacs-like bindings somewhere.
(Even in a shell, bash: "Meta-b/f" then means "move a word backward/forward".)

How to fix a mapping

In short:

See also some hints for xmodmap with a lot of comments about all this. (It's hardly readable, but should be pretty informative for the interested.)

More documentation and system specific problems

11-06-03