create new tag
, view all tags

Nedit Folding: Design Criteria

I plan to try to implement "real" folding for NEdit, with, I hope, a lot of help from others, possibly including some help from the 2008 Google Summer of Code (or 2009, or ...).

This is an attempt to define what I expect from folding in NEdit.

This has been discussed on the develop@neditPLEASENOSPAM.org list and comments have been made that should be incorporated here but that has not yet been done.



Notes on Moving this to WikiLearn

This was written to send as an attachment to the develop@neditPLEASENOSPAM.org list. Some of the notes are no longer directly relevant.

The TOC on this page is a useful tool for reading this page--read the content of the main (i.e., TWiki/HTML 2nd level) headings first, then delve into the detail if desired.

I've left the ugly terminating markup here (required for folding in kate) for no good reason, except maybe so others recognize it as ugly, as well? wink

The original askRhk (.aml) file is attached to this page so that if someone wants to try to read them in kate with folding enabled, they can do so. It will be necessary to download the kate syntax highlighting .xml file for kate 2.4.1--it may work in earlier or later versions of kate as well. It is also attached--see Attachments.


Note to the Reader

Reading this would be a lot easier in an editor that supports folding. wink You could start in the fully folded condition and then expand headings as desired to get more details.

If anybody wants it, I can supply the syntax highlighting file from kate that supports folding (developed in kate 2.4.1 for kde 3.4.2). (The file is attached, and is /usr/share/apps/katepart/syntax/askRhk04.xml on my system, and you have to tell kate to use it for files ending with the .aml extension.)



A first attempt at listing my criteria for folding. This early draft is pretty uneven--a lot of detail in places, not much in others--it would benefit from being read in a folding editor. wink

Also note:

  • Unfortunately (it shouldn't be that way), some of these criteria are phrased in the negative, i.e., what I don't want to happen. Most of those come from my recent experiences with folding in kate (KDE) and bugs I've encountered there.

  • Some of these criteria might be considered longer term goals--they might not be implemented in the first or early versions of nedit folding, but I'd like to keep them in mind so we don't preclude eventual implementation. (I know that nothing precludes eventual implementation--as much as possible I'd like to anticipate them in the initial design, so a redesign is not required.)

  • This is still a first draft, so I'm not sure I've covered all I wanted to or should cover.


Basic UI

The (g)ui should be typical of that used in similar programs for folding--that is, a column to the left of the text (the gutter or fold mark column) with special graphic characters to mark:

  • fold points in either the folded (a "boxed" "+") or unfolded (a boxed "-") state,

  • the end of fold "regions" (either an L shaped line if no folded regions overlap and continue beyond this point, or a vertical line with a "dash" to the right of the vertical line (something like an upper case "E" without the upper and lower horizontal bars, or like a plus sign without the horizontal bar to the left of the vertical bar) for an "intermediate" end of folding region), and

  • vertical lines between the boxes marking the fold points and the Es and final L marking the ends of fold regions to show the extent of a folding region

In addition, there may be a horizontal line or something else under a region of folded text to help users recognize folded (hidden) areas.

Update: yooden (Thorsten Haude, iiuc) created a gimped version of a nedit screenshot to show an alternate GUI using multiple vertical lines to show folding regions. I like it, but have a few reservations:

  • With may askRhk files, which already allow 7 (or 8) levels in the folding hierarchy, soon to be expanded to 8 (or 9) levels, and then considering that those askRhk files can contain anything, including, for example, code, which could mean another possibly 5 (just a guess) levels in a folding hierarchy, this starts to use up a fair amount of real estate. (This won't bother my working on a 21" monitor, but when I work on something less, it would probably be a problem.

  • Likewise, other people may object to the use of real estate even before getting to 14 or 15 levels in the hierarchy.

For both of these reasons, if this approach is adopted, there either needs to be an alternate or a means to show a smaller subset of those vertical lines (in some logical fashion)--just a first thought, when the cursor is in any particular folding region, show say a max of five such vertical lines usually such that one shows the extent of the folding hierarchy corresponding to this folded region, 2 show the next two folding hierarchies below this level (if any) and 2 show the next two folding hierarchies above this level (if any). The "if any" is intended to convey the idea that if the "current" folding hierarchy level is at (or near) the top or bottom of the range, there won't be two levels above or below the current level.


Means of folding / unfolding

Folding/unfolding can be accomplished by mouse clicks in the boxed "+" or "-", and also by:

  • many menu options (like: fold entire document, unfold entire document, fold all H1 headings, fold current (or selected) H1 heading(s), unfold all H1 headings, unfold current (or selected) H1 heading(s), fold all H2 headings, fold current (or selected) H2 heading(s), ..., unfold current (or selected) H6 heading(s))

This is one of the places I need to amplify/revise--the above is all described in terms of HTML headings, but the concept of a hierarchy of folding regions applies to more than just HTML (but possibly less so for programming languages??), and these menu items should either be more general (fold/unfold highest level of hierarchy, fold/unfold 2nd level of hierarchy, ...) or defined specifically for each target language.

  • keystrokes (keyboard shortcuts) to implement all of the menu options listed

  • nedit macros to implement all of the menu options listed



  • Navigation while folded will ignore folded (hidden) text.


Identifying / marking fold regions

It will be possible to use various approaches to identify / designate folding regions, including: indents, inserted markers, syntax, other?

Syntax-based fold points will be specified in a manner similar to the way syntax highlighting is specified (I think, unless we think of some better way--kate does use the same .xml file to define both syntax highlighting and fold regions/points).

Eventually (if not initially) there will be means of manually designating arbitrary fold points, (possibly by insertion of special markers--is there a better way, not requiring markers--I suppose ranges could be one of those better ways)

Eventually (if not initially) folds will be allowed within a single line (in addition to what is more common in my experience, folds that encompass only entire lines).


Folded state to (eventually) be persistent

Long term goal: the folded condition will "persist" when a file is saved, probably by use of an ancillary file storing appropriate information.


That ancillary file might not always be available for various reasons (main file sent to another computer/person and/or edited in another editor without the ancillary file), or might be out of sync with the current content of the main file.

If that ancillary file is available on opening, the folded condition as of the last save is restored.

If that ancillary file is not available, or is out-of-sync with the main file, the folded condition is not restored. (I'd like a user option to try applying the ancillary file even though out of sync--it might be a good starting point to get the file into the desired state of folding more quickly.)


Short term workarounds

Until the folded condition is persistent (as described above), these workarounds will exist (well, these workarounds will presumably always exist, once implemented, but there'd be fewer occurrences for their use)

  • A user option to have files open in the totally folded state

  • Plenty of options to fold and unfold the entire or major portions of the document (as listed under menu options, above)



Hierarchical folding

Hierarchical folding will be supported. I don't think I can explain this in 5 words or less--if necessary I'll expound more on this later--there is some discussion below.

As one specific partial example, in HTML markup, each heading would be recognized as a fold point in a hierarchy corresponding to the hierarchy of HTML headings (H1 thru H6).

Some discussion

There is one point I will mention--an approach considered at one time for folding anticipated using nedit's rangeset mechanism to mark folding areas. There was some concern expressed that such might not support hierarchical folding--iirc, the concern was (perhaps even expressed by me?) that because hierarchical folds can overlap, the rangeset mechanism might coalesce multiple fold ranges.

I think that concern was based on a (my) misunderstanding of rangesets at the time--my concern was that if folding regions in the document overlapped (which they will, see the example and discussion in "kate lookahead problem", somewhere below) the ranges might coalesce.

If my current understanding of rangesets is correct (that only ranges within the same rangeset are subject to coalescing), the solution is to have a different rangeset for each level in the folding hierarchy, (e.g., a rangeset for H1 folds, a different rangeset for H2 folds, ...), and thus there will be no concern about ranges coalescing.

Note that this is based on the assumption (I know) that folds at the same level never "meet"--they are always separated by something--in an HTML document, this would be the text of the HTML headings themselves.

Again, see the example and discussion in "kate lookahead problem", somewhere below.

Hmm, is this also true for programming languages? (I.e., that folds at the same level never meet? I'm fairly sure, as you'd usually see keywords (like if, else, case, the function header, or similar) but I'm not as sure, it may have to be considered again.)


Note re file folding_criteria.aml(.gz) not folding in kate

I forgot, and forgot to mention, that my askRhk files have an extra level in the folding hierarchy, at a level above the H1 of the HTML H1 thru H6 hierarchy.

That level is marked by a mailbox header in the form (this is an example):

From "kate: new considerations for folding" Tue Mar 04 09:01:52 2008 
Date: 03/04/08 09:01 am 
Subject: kate: new considerations for folding

The end markup, currently necessary in kate, is:


The reasons (which are not really relevant to this design criteria, but may be interesting--I will touch on one point that is or maybe relevant to the design criteria, below) for this extra level in the hierarchy include:

  • I want to allow multiple records in a single file (I could have used a much simpler header to support that, but ...)

  • I found the program recoll which indexes and then allows indexed searches of multiple files. It is capable of indexing, searching, and then displaying single emails from an mbox file. By making the askRhk header look (enough) like a real mbox header (to fool recoll), recoll can display individual records from within a multi-record file in response to search queries.

(It is slightly misleading to say recoll displays individual records, because recoll does that by using a (simple) mail program--I can't recall the name at the moment, it might be something like exmh, or maybe just mail with a one-letter prefix (ymail?).

The one point that is somewhat relevant to this design criteria

For historical reasons, I allow the first level under the From level of the folding hierarchy to be either an H1 heading (TWiki "---+ ") or an H2 heading (TWiki "---++ ").

I don't think that's a big issue, I was able to construct the kate .xml file to handle that, I will assume for now that I will be able to do the same with the NEdit means of defining folding. (If not, I'll either work on modifying it so it does, or revert to just starting the hierarchy within each record at H1.)

Hmm, I guess I should point out that with Kate I was able to define the folding points to require a strict hierarchy, that is, for example, a Level 4 heading cannot be a direct descendant of a Level 2 heading, there must be a Level 3 heading present "in between"--hmm, maybe this is some of what Bert was getting at with that last question he asked me.




Editing while folded

At times I will want editing operations to affect the entire document including any folded regions, and other times when I'd prefer that editing operations affect only non-folded regions.

There are some examples here, but more thought will be required.

  • Eventually (if not initially), you should be able to move (or insert or delete) a folded heading with or without its subheadings and subtext (if either exists).

  • You might want to search and replace something only in portions of the document--if this corresponds to the current visible (unfolded) portions of the document, this could be a convenient way to limit the scope of such replacements. On the other hand, the more normal case is probably to do the search and replace throughout the current document.

For my (askRhk) application, limiting the scope of editing operations to only the visible portion of the document is exactly what I need at least at times, and other times I need to affect the entire file, so, I very strongly see the need for both options here.

(Note to me: Sometimes that also corresponds to limiting such editing to a single or selected records in a multirecord file.)


Negative criteria, based on experience with kate

The first is a little subtle, so I definitely want to call it out, the others are pretty much glaringly obvious bugs, so maybe I wouldn't need to mention them.

kate lookahead problem

This got pretty long, so I try to summarize (not entirely accurate or complete):

Because of the design of kate neither its regex logic nor its syntax highlighting logic has been built to handle searches across multiple lines (at least in version 2.4.1 of kate for kde 3.4.2). Nor does it handle lookaheads to the next line--when it does such a lookahead, it "eats" (or moves ahead) to the location of that lookahead and can't move back.

In many cases, multiple hierarchical folding regions can end at the same point. See "discussion and example", below. Kate can't handle this except by the addition of additional markup to mark the ends of folded regions.

The need to add that additional markup sucks.

Aside: Re kate's design, iiuc, each line (ending with a newline) is stored as an independent string.

Discussion and example

Note that, because nedit operates significantly different than kate in this area (I think), the chances of this subtle bug affecting nedit seem low (I think).

Anyway, here is the bug (perhaps not described perfectly):

Note that folded regions can overlap, and thus, multiple folded regions can end at the same point. As an example, consider a situation like the following (shown using TWiki markup for headings):

 ---+ A level 1 heading
 [optional text]
 ---++ A level 2 heading
 [optional text]
 ---+++ A level 3 heading
 [optional text]
 ---+ Another level 1 heading

Note that the start of the 2nd level 1 heading marks the end of 3 folding regions--the ends of the folding regions associated with:

  • the 1st level 1 heading,
  • the level 2 heading, and
  • the level 3 heading.

Kate's regexes and syntax highlighting "logic" (which is also used for the folding logic) works (at least at this time, and in versions of kate like 2.4.1 (for kde 3.4.2)) only on a single line. Neither can deal with a \n.

(These are known bugs / design limitations in kate--I think there is a patch in the kate development "pipeline" to fix the regex, which might (or might not) lead to a speedy fix for the syntax highlighting logic.)

Ideally, what kate should do is, for example, when it is trying to find the end of the level 3 folded region, it should "lookahead" for the next heading at level 3 or higher, and end that region without "eating" the \n. It can't do that, so by the time it finds the end of the level 3 region, it has "used up" the \n preceding the ---+ marking the next level 1 region.

When the logic looks for the end of the level 2 folding region, it can't find it, because it's already been "used". Likewise for the end of the level 1 region.

(The description above is not entirely accurate, as kate doesn't (can't) look for "\n"s--instead it looks for things like "---\+{1..6}" in column 0 of a line. The principle is the same--when it finds a "---\+{1..6}" it has "eaten" too much and can't find/use the same "---\+{1..6}" to end another overlapping fold region.)

The workaround that I've found is to intentionally add ending markup. (Example below.) This sucks.

 ---+ A level 1 heading
 [optional text]
 ---++ A level 2 heading
 [optional text]
 ---+++ A level 3 heading
 [optional text]
 +++--- <added markup for end of level 3 folding region>
 ++--- <added markup for end of level 2 folding region>
 +--- <added markup for end of level 1 folding region>
 ---+ Another level 1 heading
Let's avoid this problem from the beginning.

Note to me: While writing this, I came up with at least one other approach to try in the kate .xml file which might let me do away with the markup for the ends of folding regions. I've got that documemented in another record titled "kate: new considerations for folding".


Flakey cursor when folding is in effect

The cursor (insertion point) becomes flakey when folding is in effect (when the syntax file for the current language includes specifications for folding points)--it erratically disappears--sometimes it reappears later, sometimes not.

Actually, I'm not sure, but the flakey cursor may occur even with just a plain syntax highlighting (.xml) file in effect.


Folding marker column (gutter) not in sync with text--folding works incorrectly (or not at all)

Sometimes the folding marks in the folding marker column (gutter) do not match the position of the text in any of a variety of ways--the marks may be on the wrong lines, they may show a boxed "+" which should indicate a folded condition instead of a boxed "-", the L (or modifed E) which should show the end of a folded region may be missing or at the wrong place, and similar. When the folding marks and text are out of sync, folding no longer works properly.

(Aside, I have found various ways that sometimes work to get the text and folding marks back in sync, things like scrolling to a different page and back, cutting out portions of text and then pasting them back in exactly the same position, inserting or deleting blank lines at various locations, and similar. When nothing else works, reloading the file does work every time (so far). Unfortunately, since the folded condition does not persist in kate, I then have to play around to get the file back into the folded condition suitable to what I'm doing at the time.)

I hope we don't have a similar problem, but if it arises, maybe there could be a function like "refresh (resynchronize) fold marker column" which would sort of recalculate the location and condition of fold markers as if the file were reloaded (but without disturbing the current folded condition). Probably not real easy to do--another incentive to avoid the problem wink




  • () RandyKramer - 04 Mar 2008
  • If you edit this page: add your name here; move this to the next line; and if you've used a comment marker (your initials in parenthesis), include it before your WikiName.

Revision Comments

  • R2: rhk: fixing some formatting issues I missed (and one re: bullets that puzzles me--look for a comment in the text)
  • R3: rhk: small formatting or wording fixes
  • R4: rhk: attached corrected folding_criteria.aml(.gz) file and added notes about it
  • R5: rhk: added screenshot to show "L" vs. modified "E" or "+"--need to add explanation--later
  • R6: rhk: just some wording changes to clarify meanings--more are needed wink
  • R7: rhk: wordsmithing, iirc (this log entry after the fact)
  • R8: rhk: added a link to yoodens gimped screenshot (alternate GUI), made a note about comments made on the develop list not yet incorporated here, and more wordsmithing


  • folding_criteria.aml.gz: Revised file containing the (earlier) content of this page, now with two records in the file so it does indeed fold in kate if the .xml file is applied.

  • askRhk04.xml.gz: A kate (v. 2.4.1) syntax highlighting .xml file to fold askRhk documents.

Page Ratings

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2008-03-16 - RandyKramer
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding WikiLearn? WebBottomBar">Send feedback
See TWiki's New Look