Lines versus lines in the terminal

Question

There seems to be some concept of strings and strings in terminal emulators that I would like to know more about.

Demonstration of what I mean by lines vs lines

In the Python script below, three lines of "a" and expectations are displayed, then with three lines of "b".

import sys, struct, fcntl, termios write = sys.stdout.write def clear_screen(): write('\x1b[2J') def move_cursor(row, col): write('\x1b['+str(row)+';'+str(col)+'H') def current_width(): #taken from blessings so this example doesn't have dependencies return struct.unpack('hhhh', fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, '\000' * 8))[1] clear_screen() for c in 'ab': #clear_screen between loops changes this behavior width = current_width() move_cursor(5, 1) write(c*width+'\n') move_cursor(6, 1) write(c*width+'\n') move_cursor(7, 1) write(c*width+'\n') sys.stdout.flush() try: input() # pause and wait for ENTER in python 2 and 3 except: pass 

If you reduce the width of the terminal window by one character during this break, you will see

This seems pretty reasonable - each line has been individually wrapped. When we press Enter again to print b s,

Everything works as expected. I used absolute cursor positioning and wrote to the same lines that I wrote earlier, which, of course, does not overwrite all a, because many of them are on other lines.

However, when we narrow the window by another character, the wrapper works differently:

Why did the second and third lines of b wrap together and why did the last line of a merge with the first line of b? A hint about why is in the upper visible line above - we see two a , because they are connected by two lines - of course, if we move the window again, one line will continue to be wound the same way. It seems that this happens even for strings that we replaced with a whole series.

It turns out that the lines that were wrapped earlier are now associated with the corresponding parent lines; it’s more obvious that they belong to the same logical line when we expand the terminal a lot:

My question

In practice, my question is how to prevent or predict these strings-being-merged into strings. Clearing the entire screen eliminates the behavior, but it would be nice to do this only for the individual lines that need it, if possible, so I can save caching by line, which greatly speeds up my application. Clearing to the end of the line disconnects this line from the line below, but clearing to the beginning of the line does not separate this line from the one above it.

I'm curious what that thing is? Where can I read about them? Can I find out which lines are part of the same line?

I observed this behavior with terminal.app and iterm, with and without tmux. I think that source diving in any of them will give an answer, even if there is no specification, but I suppose there is spec somewhere!

---

Background: I would like to make a terminal interface that can predict how the terminal will wrap if the user reduces the window width. I know about things like full screen mode ( tput smcup or python -c 'print "\x1b[?1049h" "that uses ncurses), which will work to prevent line python -c 'print "\x1b[?1049h" , but doesn't want to use it here.

Edit: it became clear that I already understand the script rewriting behavior and want to explain the wrapping behavior.

Answer 1

OK Start with the reasons you see:

I checked your code and noticed that this only happened when the window was resized. When the window remained at rest, it would write out a, and after pressing enter they would rewrite them with b (I assume that the expected behavior).

What seems to be happening is that when you partially resize the window, the row indices change, so at the next iteration you cannot trust the same coordinates when you call move_cursor ().

Interestingly, when you resize the window, word wrap pushes the text in front of the cursor. I assume this is part of the terminal emulator code (since we almost always want to keep the focus on the cursor, and if the cursor is at the bottom of the screen, resizing can obscure it beyond the window height if the wrap word pushed it down).

You will notice that after resizing, when you press the enter button, only two lines (and not all 3) remain visible. Here's what happens:

First, start with the initial output. (line numbers added for clarity)

 1 2 3 4 5 aaaaaaaaaaaaaaa\n 6 aaaaaaaaaaaaaaa\n 7 aaaaaaaaaaaaaaa\n 8 

Please note that at the end of each of these lines there is a new line symbol (therefore, your cursor appears below the last, despite the fact that you have not moved the cursor again)

When you reduce the window by one character, this happens:

 1 2 aaaaaaaaaaaaaa 3 a\n 4 aaaaaaaaaaaaaa 5 a\n 6 aaaaaaaaaaaaaa 7 a\n 8 

You’ll notice what I mean by pushing the text up.

Now, when you press enter and your loop repeats, the cursor goes to line 5 col 1 (as indicated by your code) and is placed directly above the last a of the second line. When he starts writing b, he overwrites the last a of the second line with b and the next line.

 1 2 aaaaaaaaaaaaaa 3 a\n 4 aaaaaaaaaaaaaa 5 bbbbbbbbbbbbbb\n 6 bbbbbbbbbbbbbb 7 bbbbbbbbbbbbbb\n 8 

It is important to note that this also overwrites the newline at the end of the second line. This means that now there is no new line separating the second line a and the first line b, so when you expand the window: they are displayed as one line.

 1 2 3 4 5 aaaaaaaaaaaaaaa\n 6 aaaaaaaaaaaaaabbbbbbbbbbbbbb\n 7 bbbbbbbbbbbbbbbbbbbbbbbbbbbb\n 8 

I'm not quite sure why this second line of b is also concatenated, but it seems to have something to do with the fact that the line in which the first is overwritten now skips its own newline completion, however, this is just an assumption.

The reason you get two line break characters if you try to squeeze the window with another character is because you are now trimming two halves of the same line of text, which means that one clicks on the other, causing two characters instead of one at the end.

For example: in these test windows, which I showed, the width starts with 15 characters, then I reduce it to 14 and print the letter b. There is another line with a length of 15 characters, and now a line of 14 a and 14 b, which is wrapped with lines with 14 characters. The same (for some reason) applies to the last two lines of b (they are one line of 28 characters wrapped in 14). Therefore, when you shorten the window with another character (up to 13): the first line of 15 a now has two trailing characters (15 - 13 = 2); the next line of 28 characters should now be in the 13-character window (28/13 = 2 R2), and the same applies to the last b.

 0 aaaaaaaaaaaaa 1 aa\n 2 aaaaaaaaaaaaa 3 abbbbbbbbbbbb 4 bb\n 5 bbbbbbbbbbbbb 6 bbbbbbbbbbbbb 7 bb\n 8 

Why does this work as follows:

Such things are the difficulty that you encounter when you try to run your program in another program, which has the right to move the text at its discretion. If you resize, your indexes become unreliable. Your terminal emulator tries to process the rearrangement for you and presses the text in front of your tooltip (which is fixed on line 8) up and down in the scroll bar so that you can always see your active invitation.

Rows and columns are something defined by a terminal / terminal emulator, and it should interpret their location accordingly. When appropriate escape sequences are specified, it is a terminal that interprets them accordingly for proper display.

Please note that some terminals behave differently, and an emulated terminal often uses a parameter to change which terminal it simulates, which can also affect the response of certain control sequences. This is why the UNIX environment usually has a parameter or environment variable ($ TERM) that tells it what type of terminal it is communicating with, so that it knows which control sequences to send.

Most terminals use standard ANSI compatible escape sequences or DEC VT hardware-based terminal systems.

In the settings of Terminal.app in the section "Settings" → "Settings"> "Advanced" you can see (or change) what type of terminal is emulated by your window in the drop-down menu next to "Declare terminal as:"

How to overcome this:

You may be able to reduce this by keeping the last known width and checking if the change has occurred. In this case, you can change the cursor logic to compensate for the changes.

Alternatively, you can use escape sequences designed to relative move the cursor (as opposed to absolute) to avoid accidentally overwriting previous lines after resizing. It is also possible to save and restore specific cursor locations using only escape sequences.

 Esc[<value>A Up Esc[<value>B Down Esc[<value>C Forward Esc[<value>D Backward Esc[s Save Current Position Esc[u Restore Last Saved Position Esc[K Erase from cursor position to end of line 

However, you have no real guarantee that all terminal emulators will deal with window resizing in the same way (which is not part of any terminal standard, AFAIK) or that it will not change in the future. If you are hoping to create a real terminal emulator, I suggest that you first get the customization of your GUI so that you can control all the resizing logic.

However, if you want to run a terminal emulator window and deal with window size reduction for this command-line utility that you are writing. I would suggest looking at the curses library for python. This is the kind of functionality that is used by all programs that support windows that I know from the top of my head (vim, yum, irssi), and can deal with such changes. Although I personally have no experience using it.

It is available for python through the curses module.

(and please, if you plan to redistribute your program, consider writing it in Python 3. Do this for children: D)

Resources:

These links may be helpful:

• ANSI Escape Sequences
• Escape Sequences VT100

I hope this helps!

Answer 2

As pointed out by 0x783czar, the key difference is whether an explicit line of a new line was printed, due to which the terminal started a new line, or there was an implicit overflow, since there were no rights to print the desired characters on the right.

It is important to remember this at the end of each line for copying purposes (whether it is a newline character in the buffer or not), for the behavior of triple-clicking on many terminals and for overriding the contents when the window is resized (in those terminals that support it) .

Applications running inside terminals almost never care about this difference, and they use the words “string” and “string” interchangeably. Therefore, when we introduced reordering of content when resizing in the gnome terminal, we preferred the words “line” or “line” for one visual line of the terminal and the word “paragraph” for content between two adjacent newline characters. A paragraph is wrapped in several lines if it is wider than the terminal. (This is by no means an official terminology, but IMO is quite reasonable and helps to talk about these concepts.)