12.3. Preventing Cross Site Scripting Vulnerabilities¶

12.3.2.1. Overview of Contexts¶

The following diagram provides a high-level overview of the relationship between the different templates, different contexts of DOM creation and manipulation, and different types of escaping. As a general rule, proper escaping is related to the context in which the data is being written, and might not match the context that will eventually be reading the data.

In the Open edX platform, data flows from the application to the initial HTML page mainly through the use of Mako templates.

Descriptions of each numbered arrow in the diagram follow.

1. This step represents the use of Mako templates to write general HTML tags (that is, tags other than <script> tags) to create the HTML page. Any data written to the page inside one of these HTML tags or HTML attributes must be HTML-escaped to be treated as plain text.
2. This step represents the use of Mako templates to write JavaScript inside a <script> tag in the HTML page. Data written to this context must all be JavaScript-escaped to keep it from mistakenly being treated as HTML. Note that this data should not be HTML-escaped, which must happen in a later step if it is written back to the DOM by JavaScript.
3. This step represents the loading and executing of JavaScript from the HTML page by the JavaScript engine. This step should be safe, if data was properly JavaScript-escaped earlier.
4. In this step, JavaScript might load additional data from HTML tags and attributes using the DOM. Sometimes data is passed in this way via data attributes. This data is typically read using jQuery functions, but it can be read using other JavaScript functions. The data can be in the form of plain text strings or JSON. If it is in the form of strings, this data should mostly be plain text. Be careful with the escaping in this case. Although the data is used in JavaScript, it is transmitted as HTML, and so must be HTML-escaped.
5. In this step, JavaScript is being used to edit the DOM, often by creating HTML tags or setting HTML attributes. Often this is done using jQuery functions. Since HTML tags and attributes are being written here, any plain text must be properly HTML-escaped.
6. This step represents a subset of DOM manipulation using JavaScript, specifically through use of Underscore.js templates. Although these templates have a specific syntax for escaping, because HTML tags and attributes are being written any plain text must be properly HTML-escaped.

12.3.2.2. HTML Context and Escaping¶

The outermost context of an HTML file is HTML. In an HTML context inside an HTML file, data is kept safe by HTML-escaping.

<div>${course_name}</div>

<script>alert('XSS attack!');</script>

<div><script>alert('XSS attack!');</script></div>

<%page expression_filter="h"/>
...
<div>${course_name}</div>

<div>&lt;script&gt;alert(&#39;XSS!&#39;);&lt;/script&gt;</div>

12.3.2.3. JavaScript Context and Escaping¶

The outermost context of a JavaScript file is JavaScript. An HTML file also contains a JavaScript context inside a <script> tag. Inside a JavaScript context, data is kept safe by JavaScript-escaping.

<script type="text/javascript">
   var courseName = "${course_name}";
   ...
</script>

";alert('XSS attack!');"

<script type="text/javascript">
   var courseName = "";alert('XSS attack!');"";
   ...
</script>

<%! from openedx.core.djangolib.js_utils import js_escaped_string %>
...
<script type="text/javascript">
   var courseName = "${course_name | n, js_escaped_string}";
   ...
</script>

<script type="text/javascript">
   var courseName = "\u0022\u003Balert(\u0027XSS attack!\u0027)\u003B\u0022\u0022\u003B";
   ...
</script>

12.3.2.4. CSS Context and Escaping¶

The browser treats any code inside a <style> tag or style attribute in an HTML page as a CSS context, or something that requires CSS parsing. CSS parsing has its own rules, and requires CSS-escaping.

In a CSS context, the following additional constraints are required to keep user supplied data safe.

• User supplied data can only appear as the value of a style property. In other words, never allow a user to supply the entire contents of the style tag or style property, or anything outside of the limited scope of an individual property value.
• User supplied URLs must use one of these safe protocols: “http:”, “https:”, or “”. Doing so prevents users from being able to supply a URL that uses the “javascript” protocol as an example.
• User supplied style property values must not contain expression(...) due to an IE feature that would enable arbitrary JavaScript to run.

There are no existing helper functions for these additional constraints in the platform. If you need to use user supplied data in a CSS context, you must work with edX to help expand the suite of available helpers.

For more information, see OWASP: CSS and XSS.

12.3.2.5. URL Context and Escaping¶

URLs require multiple types of escaping. This typically involves URL-escaping in addition to either HTML-escaping or JavaScript-escaping.

There are many special characters that are meaningful in a URL. For example, both & and = are used to designate parts of the query string. If data is being provided as a query parameter, and it might contain special characters, it must be fully URL-escaped. This is especially true with user provided data, which can contain any character. Using the JavaScript URL- escaping functions as an example, you would use the encodeURIComponent function on the data which will URL-escape all special characters. Here is an example.

var url = "http://test.com/?data=" + encodeURIComponent(userData)

URL-escaping is susceptible to double-escaping, meaning you must URL-escape its parts exactly once. It is best to perform the URL-escaping at the time the URL is being assembled.

Additionally, you will typically HTML-escape or JavaScript-escape a URL following the same rules for any other data added to the page, since a properly URL-escaped URL might still contain characters that are meaningful in an HTML context, such as & and '.

For example, when a URL is added to the href attribute of an anchor tag (<a>), it should already be properly URL-escaped, and in addition needs to be HTML-escaped at the time it is added to the HTML. When you see & between query parameters as an & in your HTML page source, you can rest easy.

When an entire URL (rather that only some query parameters) is user provided, you must also validate the URL to make sure it uses a whitelisted or acceptable protocol, such as https. Doing so prevents users from being able to supply a URL that uses the “javascript” protocol as an example.

For more information, see OWASP: URL Escape.

12.3.3.1. Django Template Files¶

Django templates are considered “safe by default”, meaning that expressions are HTML-escaped by default. HTML-escaping is not always the right choice for escaping, for example, with embedded JavaScript.

If you need to do special escaping for internationalization or a JavaScript context in a Django template, you will need to follow the patterns detailed in Mako Template Files, but we don’t currently offer documented helper functions or syntax for Django templates.

12.3.3.2. Mako Template() Calls in Python Files¶

If a Mako template is loaded from Python outside of the general template loading scheme, the following default filters should be provided to make the template safe by default (i.e. use HTML-escaping by default).

template = Template(" ... ",
    default_filters=['decode.utf8', 'h'],
)

12.3.3.3. Mako Template Files¶

This topic covers the best practices for protecting Mako template files from XSS vulnerabilities.

To convert a legacy Mako template to be safe by default, it is recommended that you complete the following steps.

1. Read through the subtopics in this section and become familiar with the current best practices.
2. Follow the step-by-step instructions detailed in Making Legacy Mako Templates Safe by Default, which will often refer back to this section.

<%page expression_filter="h"/>

<%page expression_filter="h"/>
...
${data}

<%page expression_filter="h"/>
...
<div
    data-course-name='${course.name}'
    data-course-options='${json.dumps(course.options)}'
></div>

<%page expression_filter="h"/>
<%!
from django.utils.translation import ugettext as _
%>
...
${_("Course Outline")}

<%page expression_filter="h"/>
<%!
from django.utils.translation import ugettext as _

from openedx.core.djangolib.markup import HTML, Text
%>
...
${Text(_("Click over to {link_start}the home page{link_end}.")).format(
    link_start=HTML('<a href="/home">'),
    link_end=HTML('</a>'),
)}

<%page expression_filter="h"/>
from openedx.core.djangolib.markup import HTML
...
${HTML(get_course_date_summary(course, user))}

<%namespace name='static' file='static_content.html'/>
<%!
from openedx.core.djangolib.js_utils import (
    dump_js_escaped_json, js_escaped_string
)
%>
...
<%static:require_module module_name="js/course_factory" class_name="CourseFactory">
    CourseFactory({
        course_name: '${course.name | n, js_escaped_string}',
        course_options: ${course.options | n, dump_js_escaped_json},
        course_max_students: ${course.max_students | n, dump_js_escaped_json},
        course_is_great: ${course.is_great | n, dump_js_escaped_json},
    });
</%static:require_module>

<%page expression_filter="h"/>
...
<%def name="require_module(module_name, class_name)">
    <script type="text/javascript">
        ...
        ${caller.body() | n, decode.utf8}
        ...
    </script>
</%def>

12.3.3.4. JavaScript Files¶

JavaScript files are often used to perform DOM manipulation, and must properly HTML-escape text before inserting it into the DOM.

The UI Toolkit introduces various StringUtils and HtmlUtils that are useful for handling escaping in JavaScript. You can declare StringUtils and HtmlUtils as dependencies using RequireJS define, as seen in the following example.

define(['backbone',
        'underscore',
        'gettext',
        'edx-ui-toolkit/js/utils/string-utils',
        'edx-ui-toolkit/js/utils/html-utils'],
    function (Backbone, _, gettext, StringUtils, HtmlUtils) {
        ...

If you are working with code that does not use RequireJS, then this approach will not be possible. In this situation you can access these functions from the global edx namespace instead. For more information, see JavaScript edx Namespace.

The following HtmlUtils functions all make use of HtmlUtils.HtmlSnippet. An HTML snippet is used to communicate to other functions that the string it represents contains HTML that has been safely escaped as necessary.

The HtmlUtils.ensureHtml() function will ensure you have properly escaped HTML by HTML-escaping any plain text string, or simply returning any HTML snippet provided to it.

If you must perform string interpolation and translation, and your string does not contain any HTML, then use the plain text StringUtils.interpolate() function as follows. This function will not escape, and follows the best practice of delaying escaping as late as possible. Since the result is a plain text string, it would properly be treated as unescaped text by any of the HtmlUtils functions.

StringUtils.interpolate(
    gettext('You are enrolling in {courseName}'),
    {
        courseName: 'Rock & Roll 101'
    }
);

If you are performing string interpolation and translation with a mix of plain text and HTML, then you must perform HTML-escaping early and the result can be represented by an HTML snippet. Use the HtmlUtils.HTML() function to wrap any string that is already HTML and must not be HTML-escaped. The function HtmlUtils.interpolateHtml() will perform the interpolations and will HTML-escape any plain text and not HTML-escape anything wrapped with HtmlUtils.HTML(). See the following example.

HtmlUtils.interpolateHtml(
    gettext('You are enrolling in {spanStart}{courseName}{spanEnd}'),
    {
        courseName: 'Rock & Roll 101',
        spanStart: HtmlUtils.HTML('<span class="course-title">'),
        spanEnd: HtmlUtils.HTML('</span>')
    }
);

You can also use HtmlUtils.joinHtml() to join together a mix of HTML snippets and plain text strings into a larger HTML snippet where each part will be properly HTML-escaped as necessary. See the following example.

HtmlUtils.joinHtml(
    HtmlUtils.HTML('<p>'),
    gettext('This is the best course.'),
    HtmlUtils.HTML('</p>')
)

Often, much of the preparation of HTML in JavaScript can be written using an Underscore.js template. The function HtmlUtils.template() provides some enhancements for escaping. First, it makes HtmlUtils available inside the template automatically. Also, it returns an HTML snippet so that other HtmlUtils functions know not to HTML-escape its results. It is assumed that any HTML-escaping required will take place inside the Underscore.js template. Follow the best practices detailed in Underscore.js Template Files.

The final step of DOM manipulation in JavaScript often happens using JQuery. There are some JQuery functions such as $.text(), $.attr() and $.val() that expect plain text strings and take care of HTML-escaping its input for you.

There are other JQuery functions such as $.html(), $.append() and $.prepend() that expect HTML and add it into the DOM. However, these functions do not know whether or not they are being provided properly escaped HTML as represented by an HTML snippet.

If you are working with a Backbone.js element, as represented by el or $el, you can use the JQuery methods directly, as in the following example.

this.parentElement.append(this.$el);

However, if you are creating the element through one of the other HtmlUtils functions, you must use HtmlUtils.setHtml(), HtmlUtils.append() and HtmlUtils.prepend() in place of the JQuery equivalents. These HtmlUtils JQuery wrappers respect HTML snippets, and can be used as seen in the following example.

HtmlUtils.setHtml(
    this.$el,
    HtmlUtils.joinHtml(
        HtmlUtils.HTML('<p>'),
        gettext('This is the best course.'),
        HtmlUtils.HTML('</p>')
    )
);

In the case of Backbone.js models, although attributes can be retrieved using the get() or escape() functions, you should avoid using the escape() function, which will HTML-escape the retrieved value. It is preferable to use the get() function and delay escaping until the time of rendering, which is often handled using an Underscore.js template.

To properly URL-escape, you can use the JavaScript functions encodeURI and encodeURIComponent. The following example shows how to properly URL-escape user provided data before it is used as a query parameter.

var url = "http://test.com/?data=" + encodeURIComponent(userData)

For more information about URLs, see URL Context and Escaping.

edx.StringUtils.interpolate(...);
edx.HtmlUtils.setHtml(...);

12.3.3.5. CoffeeScript Files¶

For CoffeeScript files, follow the same guidelines as provided for JavaScript files, but using the CoffeeScript syntax.

12.3.3.6. Underscore.js Template Files¶

The best way to HTML-escape expressions in an Underscore.js template is to use the <%- tag, which will perform the HTML-escaping.

There are some exceptions where you must use a combination of <%=, which does not escape, and one of the UI Toolkit HtmlUtils functions. One example is when you use the HtmlUtils.interpolateHtml() function to translate strings that consist of a mix of plain text and HTML. You can easily gain access to the HtmlUtils object inside a template by rendering the Underscore.js template using the HtmlUtils.template() function.

If you need to pass an HTML snippet to a template, which has already been HTML-escaped, you should name the variable with an Html suffix, and use HtmlUtils.ensureHtml() to ensure that it was in fact properly HTML- escaped. See the following example.

<%= HtmlUtils.ensureHtml(nameHtml) %>

For more details about using the HtmlUtils utility functions, see JavaScript Files.

12.3.4.1. Set HTML-Escaping Filter as Default¶

Start by adding the following line to the very top of your Mako template.

<%page expression_filter="h"/>

It is important to understand that this change will affect all expressions in your Mako template. Although HTML-escaping is a reasonable default, it might cause issues for certain expressions, including HTML that cannot be escaped.

Also, be careful not to have multiple <%page> tags in a Mako template.

12.3.4.2. Run the Safe Template Linter¶

After setting HTML-escaping by default for the Mako template, run the Safe Template Linter with the following command.

./scripts/safe_template_linter.py

Accuracy and completeness of the linter are not guaranteed, so test your work after fixing all violations.

For more detailed instructions on using the linter, see Safe Template Linter.

12.3.4.3. Fix Downstream JavaScript and Underscore.js Templates¶

Because Mako templates only generate the initial page source, you should ensure that any downstream JavaScript files or Underscore.js templates also follow the best practices.

When you have found the proper downstream JavaScript and Underscore.js template files, you can again run the Safe Template Linter on these files.

For help navigating our client side code, see Navigating JavaScript and Underscore.js Templates

12.3.5.1. Disabling Violations¶

If you need to disable a violation, add the following disable pragma to a comment at the start of the line before the violation, or at the end of the first line of the violation. Use the comment syntax appropriate to the file you are editing.

Here is example syntax for a Mako template.

## safe-lint: disable=mako-invalid-js-filter,mako-js-string-missing-quotes

Here is example syntax for an Underscore.js template.

<% // safe-lint: disable=underscore-not-escaped %>

12.3.5.2. Linter Violations¶

The following topics explain the meaning of each violation ID and what you must do to resolve each violation.

// Do NOT do this
self.$el.append(
    _.template(teamActionsTemplate)({message: message})
);

// DO this
HtmlUtils.append(
    self.$el,
    HtmlUtils.template(teamActionsTemplate)({message: message})
);

// DO this
self.$el.append(
    HtmlUtils.template(teamActionsTemplate)({message: message}).toString()
);

// DO this
self.$el.append($button);

// Do NOT do this
this.button.after(message);

// DO this
messageHtml = HtmlUtils.template(messageTemplate)({message: message});
this.button.after(
    HtmlUtils.ensureHtml(messageHtml).toString()
);

## Do NOT do this
<%page expression_filter="h"/>
...
${_("Details &amp; Schedule")}

## DO this
<%page expression_filter="h"/>
...
${_("Details & Schedule")}

## DO this sparingly
${HTML(x)}
## or
${x | n, decode.utf8}

<%!
from openedx.core.djangolib.js_utils import dump_js_escaped_json
%>
...
${x | n, dump_js_escaped_json}

## or
<%!
from openedx.core.djangolib.js_utils import js_escaped_string
%>
...
${x | n, js_escaped_string}

## or DO this sparingly
${x | n, decode.utf8}

// Do NOT do this
var invalid = '<strong>${x | n, js_escaped_string}</strong>'

// DO this
var valid = '${x | n, js_escaped_string}'

// Do NOT do this
var message = ${msg | n, js_escaped_string}

// DO this
var message = '${msg | n, js_escaped_string}'

<%page expression_filter="h"/>

## Do NOT do this
<%page expression_filter="h" />
...
<%page args="section_data" />

## DO this
<%page args="section_data" expression_filter="h" />

## Do NOT do this
${Text(_("Click over to {link_start}the home page{link_end}.").format(
    link_start=HTML('<a href="/home">'),
    link_end=HTML('</a>'),
))}

## DO this
${Text(_("Click over to {link_start}the home page{link_end}.")).format(
    link_start=HTML('<a href="/home">'),
    link_end=HTML('</a>'),
)}

# Do NOT do this
msg = '<html>' + msg + '</html>'

# DO this
msg = HTML('<html>{msg}</html>').format(msg=msg)

## Do NOT do this
${_("Click over to {link_start}the home page{link_end}.").format(
    link_start=HTML('<a href="/home">'),
    link_end=HTML('</a>'),
)}

## DO this
${Text(_("Click over to {link_start}the home page{link_end}.")).format(
    link_start=HTML('<a href="/home">'),
    link_end=HTML('</a>'),
)}

12.3.6.1. Why Use Both js_escaped_string and dump_js_escaped_json?¶

To escape strings in Mako templates, why must we use dump_js_escaped_json in addition to using js_escaped_string?

• The js_escaped_string function provides the additional benefit of returning an empty string in the case of None.
• The js_escaped_string and wrapping quotes makes the expected type more declarative.

12.3.6.2. Mako Filter Ordering and the n Filter¶

Mako executes any default filter before it executes filters that are added inside an expression. One such default filter is the decode.utf8 filter, which is used to decode to UTF-8, but only if the Python object is not already unicode.

Take the following example Mako expression.

${data | h}

When Mako compiles this expression to Python, it is translated to the following Python code.

__M_writer(filters.html_escape(filters.decode.utf8(data)))

From the Python line above, you can see that the default decode.utf8 filter is applied before the the h filter, which was supplied inside the expression.

The n filter can be used to turn off all default filters, including the decode.utf8 filter. Here is an example Mako expression.

${data | n}

In this case, when Mako compiles this expression to Python, the following Python code is the result.

__M_writer(data)

For more information, see Mako: Expression Filtering.

12.3.6.3. Mako Blocks¶

A Mako %block can sometimes create tricky situations where the context is not clear. In these cases, it would be best to provide the context (for example, HTML or JavaScript) in the name of the block.

Take the following Mako %block definition as an example.

<%page expression_filter="h"/>
...
<%block name="html_title">${display_name}</%block>

Based on the above %block definition, only the name of the block tells us that it is HTML-escaped, and it is only usable in an HTML context. You could not use this same %block in a JavaScript context.

Here is this same %block above, as it is actually used to display the title.

<title>
    <%block name="html_title"></%block>
</title>

For more information, see Mako: Defs and Blocks.

12.3.6.4. Strings Containing JSON in Mako¶

In the same way that we wait as long as possible to escape, once we know the context, we also recommend waiting as long as possible before converting from Python to JSON. Mako templates are often the place where the Python object should finally be dumped to JSON.

If you find yourself with a string that already contains JSON inside a Mako template, and you need to use it in a JavaScript context, you have the following two options.

• Where appropriate, you could attempt to refactor the code to move the call to json.dumps from the Python file feeding the Mako template, into the Mako template, replacing that call with dump_js_escaped_json.
• You can call json.loads before dumping it to ensure it is parseable, as in the following example.

<script>
    var options = ${json.loads(options_json_string) | n, dump_js_escaped_json};
</script>

12.3.6.5. Why Do I Need Text() with HTML()?¶

You might wonder why the Text() function is required in Mako templates to make the HTML() function work.

The magic behind the Text() and HTML() functions is a library called markupsafe and its Markup class, which designates that a string is HTML markup and no longer needs to be HTML-escaped. The difference between Text() and HTML() is that Text() HTML-escapes before it becomes Markup, where HTML() simply marks a string as Markup.

The magic of Markup is that any string that is formatted into it is HTML- escaped during that process. Note how the & is HTML-escaped in the following example.

>>> from markupsafe import Markup
>>> Markup('<div>{}</div>').format('Rock & Roll')
Markup(u'<div>Rock &amp; Roll</div>')

For the next example, when Markup is formatted into a Markup object, it understands that it should not be HTML-escaped and thus the & will remain unchanged.

>>> Markup('<div>{}</div>').format(Markup('Rock & Roll'))
Markup(u'<div>Rock & Roll</div>')

A problem arises when we use format on a plain string. Since a string does not know anything about Markup, the result of this is a plain string again, rather than a Markup object. Thus, the result has lost its Markup magic.

>>> '<div>{}</div>'.format(Markup('Rock & Roll'))
'<div>Rock & Roll</div>'

In Mako, we add page-level HTML-escaping by default, which also uses the markupsafe library. Mako expressions will therefore respect Markup objects and will not double escape.

<%page expression_filter="h"/>
...
${data}

Therefore, the problem with using HTML() without the initial Text() is that the Markup object becomes a plain old string and it ends up getting HTML-escaped, when your intention was to keep the HTML from being HTML-escaped.