Why A New Language?

One could argue that the goals of the HFL language can be achieved by many of the current plethora of web design tools.

The perception can be: what does it matter if HTML and CSS are messy and hard to work with when one can use a web tool to make them less of a problem? Who needs a new language to fix this?

What is wrong with the current tools to build websites?

There are a very large number of tools in the form of IDEs (Integrated Development Environments) such as DreamWeaver, and content-management systems such as Joomla!, that can greatly reduce the effort needed to create web pages.

The problem is that they require that their users to be skilled web designers. While they can make manual construction of a site easier, they do not help with the real problem: making good artistic decisions.

Most of them are WYSIWYG (What You See Is What You Get) graphical user interfaces, which present the layout visually as you create it. Content-management systems (CMSs) allow site owners to make certain types of modifications to an existing site themselves, rather than have to summon a professional to do it for them.

Many content-management systems can load content into pre-defined slots in a layout template, which holds the promise of being able to swap layout looks. But the results will not look very professional unless each page is individually adjusted.

A professional-looking site requires making artistic layout decisions that are aesthetically pleasing.

Such results are created by fine-tuning the layout. This involves adjusting component positions, display attributes and behavior based upon the combinations of actual content. The presence or absence of layout elements can affect the layout and possibly the choice of display properties of the other elements, and often the HTML structure around them. To make the overall look attractive requires intuition as to what will be most effective.

That requires that the designer have the skills usually associated with a graphic artist. And therein lies the problem.

Most site owners are not graphic artists. If they wish to create a professional-looking site, they must hire a graphic artist to create it, and possibly maintain it. Otherwise they risk having a site that is rather bland.

How can a software tool help a user make good artistic decisions?

A site owner might not be gifted artistically, but still can know what works and what does not when it comes to what he/she wishes to present.

What if it were possible to choose a layout look from a palette of layouts? One could try out a series of looks, one at a time, until one found one that seems best suited.

This has been attempted many times, usually where the user fills in the fields of some type of template.

But templates have a few problems.

1. In most template systems the templates contain the content, rather than refer to it. To try out different looks, the designer would have to physically copy the content from one template to another. This would be tedious and time-consuming, and it is unlikely that the designer would try out very many different looks as the cost is too high.

   Many content-management systems (CMS) do refer to some of the content, by having fixed content field names, or as references to fields within a database. This can allow the user to try out different pre-defined templates, provided that each one supports the same data or database fields. This in turn requires that there be strict standards as to what each field would be called, or else the templates will not work properly.

   Many CMS's provide some sample templates to help people get started, but they cannot have broad application, because they have to refer to specific data or database fields. And very few if any can make more than minor adjustments when some fields are empty.

   Another problem with CMS's is that the site definition is stored on a server, not in local user files, and so the content is locked into the specific CMS system. The sources can only be accessed using that CMS software, and typically are not in any standard format. To use a different site host or to use a different or no CMS, a site owner would have to port the content manually.

2. Templates by their very nature are not very flexible. Templates can only support content that fit their pre-defined fields. Even if the templates were beautifully designed by talented graphic artists, they cannot easily adapt to varying kinds of content.

   Most template systems use templates that are single structures. The components are usually located within fixed zones of a page, and the layouts can usually only account for missing content by adjusting the dimensions of the remaining content of the page. The only way to have different pages display differently is to create different templates.

   Some systems have complicated work-arounds (such as 'ifdef' constructs) to try and make their templates more flexible, but at the cost of ease of use. While professionals can use such systems, they are generally too complicated for non-professional site owners.

   Some CMS's do support hierarchical templates, where components can be controlled by sub-templates. While this does go a long way toward the concept of an a-la-carte construction of a site, each page type still needs to use a template that is for that type, resulting in multiple outer templates. And such systems still cannot automatically adjust to the actual content.

3. Template-based systems are limited to the functionality that was provided by the template program. So when an effect is needed that is not in the basic set provided by the system, it can require complex work-arounds to create it. Such systems are rarely user-extensible beyond minor features.

   Again, this does not lend itself well to building reusable layout libraries.

4. Creating usable custom templates requires considerable expertise in all of the template-based systems, and is not something that is easy to do for non-professionals. There are very few standards, so each template-based system (offline or CMS) has its own way of doing things, with its own learning curve.

The bottom line is that site owners cannot simply take their content, point it at a single template, and see that content displayed in a professional way. Template-based systems contain the content instead of pointing to it. While each individual action that a designer must do may be quite easy, the total number of decisions and actions to move content into templates requires a lot of time and expertise and makes site construction a large amount of work. So the goal of having a site owner simply choose a professional look from a palette is thwarted by the very nature of templates.

The problem is: flexible layouts are not possible unless the content is kept separate from the layout information, and that layout information is capable of adapting to the varying nature of the content.

Why describe a layout using a language instead of a template?

Where a language-based (programmable layout) system really makes a difference over a template-based system is that it can automatically adjust layout and formatting to the specific content, not just do it statically. It is far more flexible than any template could ever be.

While a template can organize the overall features of a page, a language-based layout can automatically format the content within those features. This can be a great advantage when the content being displayed was itself computer-generated, such as the output of a database query, or the output of an application program. Where a template-based system would require that all such content be displayed the same way, a language-based system can display much more refined layouts, based upon the specific content.

Programmable layout descriptions can also modify the content that will be output to create different versions. As an example, documentation files for Windows, Mac and Linux need to have different textual content and examples to account for the differences in the operating systems. Simple conditional-compile flags in some templates can help by separating different versions of the content, and some systems support simple macros for text substitution. But a script-based system can not only hide or show fixed content and do macro expansions, it can rearrange layouts based upon that content or version.

Programmable layout descriptions can also simply the task of manually entering content because they can automatically apply common display standards, relieving the web designer from having to apply them manually. An example would be enforcement of indentation rules such as first-line indent. While this sort of thing can be done manually in an IDE on a field by field basis, it can be done globally in a programmable layout. Merely switching layout descriptions can have result in global stylistic changes.

But where a language-based layout can really shine is in producing a consistent, aesthetically-pleasing result. This requires making coherent artistic decisions that are context-sensitive.

Appealing layouts are created as a set of interlaced conventions, involving position and alignment decisions, color choices, font choices, indentation rules, and rules about how various types of content are displayed next to one another. The presence or absence of specific components on a page can affect the display of the remaining components. These decisions cannot be done very well in a static layout description such as in a template. And yet can be easily described in a language-based layout.

Programmable layout descriptions also can facilitate an utterly essential requirement: being able to assemble a layout from lower-level layout components.

This allows site designers to take a base layout and tune it by trying different sub-modules to achieve their desired looks.

But creating swappable sub-layout modules creates a major problem.

It is inevitable that if different groups of people contribute to a formatting library, that they will create different versions of similar features that will be incompatible with the others.

As an example, the jQuery framework has thousands of contributions for all kinds of widgets, with a great number of variations of many of the widget types (for example, navigation bars). And very few of these variations have source requirements that are compatible with any others. To switch from using one to another usually requires modification of the source files to match the requirements of the new widget. Enforcing standards upon widget designers does not really work, because different widgets may legitimately need source content that others do not, or legitimately must be in a different form. And who decides upon the standard in the first place? What if a better widget comes along later, that must have different inputs?

This issue becomes greatly magnified when trying to create general layout libraries, and especially interchangeable sub-layout modules.

For example, different sub-layout modules may want the content to be marked with different tags or ids, or may apply tags or ids that are different than the enclosing layout description expects.

For layouts created by different people to work together, there needs to be a way to negotiate a common interface. In a conflict, the results of one of the two layouts needs to be adapted to conform to the requirements of the other.

This is possible in a programmable layout description, and generally impossible in a template-based system. Part of the layout library might need to be a set of small 'adaptor' descriptions, that modify (or 'map') the content from one form to the other. These adaptors could even be self-loading. But without the ability to build adaptors, it is not really practical to build formatting libraries that receive open contributions.

Finally, a layout description that is based upon a programming language has one major advantage over a template- or structure-based system: it is extensible. Layout designers are not limited to whatever features are provided by the system, they can create new features as needed. This means that formatting libraries can evolve and become more and more sophisticated over time.

When put all together, formatting libraries become possible and practical if the layout descriptions can have internal logic, that is are based upon a programming language, not data structures. The flexibility provided by programmability is essential to creating flexibility in the layout descriptions.

What this means is that a graphic artist, possibly working with a programmer, can create elegant and sophisticated layouts that are very flexible and can be used in a broad range of applications. Such layout descriptions would have value, and thus could be a marketable commodity. And this in turn provides incentive to create finely tuned layouts.

And that means that for a site builder, it is possible to create a site by referencing pre-defined layout descriptions, then refine them to taste.

Can't this be done in HTML, CSS and javascript?

Page layout in HTML is a combination of structure (HTML), formatting (CSS), and perhaps interactivity (javascript). Any change from one look (layout) to another will involve restructuring or creating new HTML code, and perhaps some CSS or javascript code as well. A format (programmable layout) is a program that when it executes generates that code.

Neither HTML nor CSS are programming languages, and thus cannot be the basis of a programmable-layout system. And javascript is a run-time programming language, and cannot help in the construction of the site's source files themselves.

To be effective, the layout language needs to run at compile-time, that is, be a source-level language like C. It's output will be the HTML, CSS and javascript source files of the site.

This cannot be done with the current versions of HTML, CSS and javascript.

Isn't that what server-side languages like PHP do?

Server-side languages like ASP and PHP are programming languages. They emit code as output, and thus could conceivably be used to create layout libraries.

But it is not enough that code could be written in them that could do layouts. It has to be usable by non-programmers, and have very clear semantics.

The problem is that they are clearly designed to be programming languages, first and foremost. Any layout that they do is in the form of code instructions, which look nothing like formatting descriptions. This is really not user-friendly to non-programmers.

The biggest sticking point is that one cannot simply read the their source files and get a pretty clear understanding of what the layout will be. The source files are programming instructions, and it takes a programmer to understand what they are doing.

And even programmers cannot know what the layout will be without intense study when it is expressed as ordinary code, because what the code is doing will be obscure.

None of these languages were designed to make creating layouts simple and easy to read.

If we expect non-programmers such as graphic designers and normal users to be able to modify formats, they must be easy to understand. That means that the syntactic structure of the language chosen to define the layouts is critical.

This is not a matter of programmability, it is a matter of usability. It is possible to design a language that centers around layout first, and programming second. That is why HFL was created.

What kind of layout language would be usable by graphic artists?

Just creating a programming language to do layout is not enough. That language has to be simple enough to use that web developers can use it without having to be programmers themselves.

To build an aesthetically pleasing result, many artistic decisions must be made. Layout and formatting are intertwined. Layout affects formatting; formatting affects layout. How pleasing and effective the end result will be depends greatly upon the artistic skill level of the web developer. As such, the most attractive sites are usually created by developers that are graphic artists. But rarely are graphic artists very skilled at programming.

A language designed to define layouts must not look like a programming language when used in simple ways, or it will not be usable by graphic designers. It should look more like a description of the layout. And yet it should look like a programming language to those who are programmers. Furthermore, it must not require that either group have to learn a whole new language before they can get started.

The simple answer is: To web designers, it looks like a slight enhancement of CSS. If they know CSS, they are 95% there. To a programmer, it looks like a slight enhancement of javascript. For either group, there is only a couple of syntactic elements to learn. So the learning curve is very small.

To make the language usable, designers must be able to easily swap layout components. That means that the functionality of the layout components must be independent from the context they run under.

For web designers to be able to locate and use them, the layout descriptions must be accesible by name, and thus stored in files. In HFL, these layout-description files are called 'formats', to distinguish them from data structures such as templates.

To the graphic artist, creating and using a layout component should be no harder than creating or using CSS, and be done they same way. And swapping one component with another must as simple as naming a different format (layout file).

To the programmer, logic can be added that allows formats to adjust to their content, calculate property values, generate display effects, and control which formats will be used based upon selection criteria. And the syntax to do all means that non-programmers would be able to read the code.

Anyone should be able to create an aesthetically-pleasing look for a page by selecting a pre-defined format. And anyone can tune that look by swapping formats for sub-components with others. The process of creating an initial site look is mostly that of trying different formats a-la-carte, until one finds one that is most appealing to the designer.

This requires that formats be modular and interchangeable.

The goals of the HFL language

HFL is a layout language first, and a programming language second. To achieve this, its syntax has an scope-action organization. Formats are blocks of instructions in which a range of content (or 'scope') is identified, and then one or more actions are applied to that scope (such as setting display properties).

For example, the following would set the color of the text of the HTML element marked with 'id = heading' to red:

• #heading { color: red; left: 200px;}
  

Here, the 'scope' is the HTML element of id '#heading', and the 'action' is to apply the 'color' property to 'red' and set the 'left' indent to '200px'.

Note that is exactly how you would do it in CSS. For graphic designers familiar with CSS, this should be eminently readable.

For comparison, consider how the same code might be written in javascript:

• var el = document.getElementById('heading').style.color = 'red';
  el.style.left = '200px';
  

While both of these examples say the same thing, the HFL version is terse and readable.

The javascript version is verbose and harder to read, because the eye has to sort out the scope (document.getElementById('heading')) from the action (color = 'red'). Plus it is necessary to create a local variable, and make object-oriented references to the 'style' field.

The biggest difference between the two examples is that in HFL, actions are specified inside of a scope specification similar to CSS, and in javascript the scope must be specified for every action.

That requires anyone reading the code to pay strict attention to the scopes being specified, or they may misinterpret what is being done. It is far too easy to fail to notice that a different variable was used.

Where the HFL example looks like a CSS property description, the javascript example very much looks like (and is) code. Examples written in any of the other current web programming languages would have similar appearances, because they are all fundamentally procedural languages, and thus work in the same way.

While all of that extra syntactic clutter may not bother many programmers, it is more difficult for non-programmers. And this is a very simple example.

It is far easier to understand a scope containing properties that apply to it, than to read code. That scope and CSS-style property syntax must be built into the language itself. It cannot be duplicated by coding conventions.

For a layout language to be truly practical, it must be easy to read a description and be able to visualize the HTML structure that will be generated.

One of the hardest things about a language-based design is that it can be very difficult to figure out what effect the code will have on the final layout. The very flexibility of a language can encourage creation of obscure code. In this respect, templates have an advantage because it can be easier to understand what they will do.

To address this, the syntax of HFL is inherently hierarchical. Scopes can be nested within other scopes (something CSS cannot do), which tracks the hierarchical structure of the HTML elements they are manipulating.

For more on scopes see below.

HFL combines the syntax of CSS, javascript and HTML into one language, so that there is very little new syntax to learn.

Many property assignments can be specified in CSS syntax, specified in HTML attribute syntax, or coded in javascript object syntax. HFL allows properties to be set in any of the three syntaxes, interchangeably. At any point in a format description, any of the syntaxes can be used. It is up to the format designer as to which syntax he/she chooses to use at any point.

To graphic designers, HFL looks very much like CSS, with enhancements that allow much finer distinctions and control.

To programmers, HFL looks very much like javascript, with enhancements that make it far easier to identify HTML elements and apply properties to them.

HFL is an editing and transformation language

The language is oriented around identifying content ranges (scope) and applying changes to it.

Its scope-based syntactic structures are implicit search specifications.

Its basic operations are create, insert, cut, copy, and paste. It can create, modify or rename any HTML tag or property, all under the control of scope specifications as explained below. It can therefore completely modify or replace any HTML or CSS content, including completely re-structuring and re-formatting existing pages.

It is a compiler, that is it reads in source files and emits other source files (HTML, CSS, javascript, images files and so on). What is unusual is that it does not just have some minor macro capability such as preprocessor directives. The language code itself runs at compile time and controls what code will be generated.

This solves a large array of problems in a very simple way.

1. The code is far more readable and easier to understand than any normal programming language, due to its syntactic support for scope.

2. Because HFL can construct HTML structure around content, content can be stored separately from enclosing HTML structure and CSS or HTML formatting. Because the content has little or no enclosing structure, it is far easier to create formats that work across widely different input types.

   Most HFL source files contain just the actual content (text, image references, hyperlinks, etc.) and references to one or more formats to do the layout. The user source files are typically very small, and because they are not cluttered by all of the HTML structure, usually very readable in themselves. One can easily see the various pieces that will be displayed.

   This is possible because the formatting descriptions themselves can provide whatever structure that they require to produce their effects. An HFL format usually does not need the application content to be adapted to its structure requirements, most can adapt that content as needed themselves. And that in turn makes it possible to radically change the look of websites by swapping format descriptions.

   When content is completely separate from formatting and structure, it becomes possible to create libraries of entire looks for a site, and swap one look for another at a later time. These libraries can include formats for anything from single localized effects to a complete definition of a site.

3. HFL formats can be designed that automatically generate HTML tags by scanning the source files. This can greatly reduce the complexity of source files, reducing the amount of work it takes to create them, and making them far more readable.

   For example, HTML generally requires that paragraphs be marked with <p> tags, that numbered lists be marked with <ul> tags, bullet lists with <ol>, and so on. But a single format can scan the source file and generate all of these tags automatically.

   This document was created using one such format. (To see the source for this document, click on the "Show Page Source" button on the left.)

4. HFL solves the problem of incompatible input requirements of formats created by different contributors by its ability to transform and edit content so it can conform to a format's input requirements.

   Small segments of adaptor code can be written that convert one input content arrangement into another. In many cases, the format itself would perform the transformation, or provide a small set of adaptor functions. In others, a few lines of application code can perform the transformation.

   This allows contributors complete freedom to construct their formats in the way that works best, and yet still meet the goal of have interchangeable formats.

5. In HFL it is easy to make layouts context-sensitive, because it is a programming language that supports conditional logic. Not only can is it possible to include or exclude certain components based upon various criteria, it is possible to generate entirely different structures.

   This means that it is possible to create a 'super' format that can generate entire classes of page types. The web designer would reference the same format for all pages of a site, instead of one per page type. This makes swapping one entire look for another far simpler.