A number (usually white) tells what floor a ladder, staircase, etc will take you to.
One-way transporters are marked as "To 3b", "From 4a", etc.
On floor 2 of the Castle of Ordeal, red numbers in white circles show transporters within the same floor. They show the order you need to take.
Letters (usually red) show the contents of chests. A letter that seems out of place, like an "E" on a floor with only one chest, means that the chest shares its contents with one or more chests on the same or other floors. Opening one will also empty the other(s).