loggerhead

Merge lp:~mkanat/loggerhead/limit-pygments into lp:loggerhead

  1. limit-pygments
  2. Merge into trunk-rich
Proposed by Max Kanat-Alexander
Status: Merged
Approved by: Ian Clatworthy
Approved revision: 409
Merged at revision: not available
Proposed branch: lp:~mkanat/loggerhead/limit-pygments
Merge into: lp:loggerhead
Diff against target: 32 lines (+9/-0)
1 file modified
loggerhead/highlight.py (+9/-0)
To merge this branch: bzr merge lp:~mkanat/loggerhead/limit-pygments
Reviewer Review Type Date Requested Status
Ian Clatworthy (community) Approve
Review via email: mp+23900@code.launchpad.net

Commit message

Make pygments only highlight files if they're smaller than 512K, to prevent overloading loggerhead.

Description of the change

A fix for bug 513044--limit pygments to only being used on files 512K or smaller.

To post a comment you must log in.
lp:~mkanat/loggerhead/limit-pygments updated
409. By Max Kanat-Alexander

Don't put parens around the if condition

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

The value probably ought to be a configuration setting down the track but this looks a sensible improvement as is. If anything, I'd be tempted to make the default limit even lower, e.g. 256K.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'loggerhead/highlight.py'
2--- loggerhead/highlight.py 2009-05-16 06:03:47 +0000
3+++ loggerhead/highlight.py 2010-04-22 03:26:24 +0000
4@@ -16,6 +16,8 @@
5 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
6 #
7
8+import cgi
9+
10 from pygments import highlight as _highlight_func
11 from pygments.lexers import guess_lexer, guess_lexer_for_filename, TextLexer
12 from pygments.formatters import HtmlFormatter
13@@ -23,12 +25,19 @@
14
15 DEFAULT_PYGMENT_STYLE = 'colorful'
16
17+# Trying to highlight very large files using pygments was killing
18+# loggerhead on launchpad.net, because pygments isn't very fast.
19+# So we only highlight files if they're 512K or smaller.
20+MAX_HIGHLIGHT_SIZE = 512000;
21
22 def highlight(path, text, encoding, style=DEFAULT_PYGMENT_STYLE):
23 """
24 Returns a list of highlighted (i.e. HTML formatted) strings.
25 """
26
27+ if len(text) > MAX_HIGHLIGHT_SIZE:
28+ return map(cgi.escape, text.split('\n'))
29+
30 formatter = HtmlFormatter(style=style, nowrap=True, classprefix='pyg-')
31
32 try:

Subscribers

People subscribed via source and target branches