Commit c5092d41 authored by Michael Tremer's avatar Michael Tremer

importer: Try to make parsing blocks faster

This patch changes the order of some operations and drops a few
redundant ones (potentiall).

That should allow us to parse blocked data faster.
Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
parent f35adb4c
...@@ -200,6 +200,10 @@ def iterate_over_blocks(f, charsets=("utf-8", "latin1")): ...@@ -200,6 +200,10 @@ def iterate_over_blocks(f, charsets=("utf-8", "latin1")):
block = [] block = []
for line in f: for line in f:
# Skip commented lines
if line.startswith(b"#") or line.startswith(b"%"):
continue
# Convert to string # Convert to string
for charset in charsets: for charset in charsets:
try: try:
...@@ -209,24 +213,17 @@ def iterate_over_blocks(f, charsets=("utf-8", "latin1")): ...@@ -209,24 +213,17 @@ def iterate_over_blocks(f, charsets=("utf-8", "latin1")):
else: else:
break break
# Skip commented lines
if line.startswith("#") or line.startswith("%"):
continue
# Strip line-endings
line = line.rstrip()
# Remove any comments at the end of line # Remove any comments at the end of line
line, hash, comment = line.partition("#") line, hash, comment = line.partition("#")
if comment: # Strip any whitespace at the end of the line
# Strip any whitespace before the comment line = line.rstrip()
line = line.rstrip()
# If the line is now empty, we move on # If we cut off some comment and the line is empty, we can skip it
if not line: if comment and not line:
continue continue
# If the line has some content, keep collecting it
if line: if line:
block.append(line) block.append(line)
continue continue
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment