Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
A
apachedex
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Xiaowu Zhang
apachedex
Commits
a4c6f5d7
Commit
a4c6f5d7
authored
Apr 10, 2013
by
Vincent Pelletier
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Move lengthy usage explanations to README.
parent
5c2d9a36
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
26 additions
and
23 deletions
+26
-23
README
README
+18
-0
apachedex/__init__.py
apachedex/__init__.py
+8
-23
No files found.
README
View file @
a4c6f5d7
...
@@ -130,6 +130,16 @@ A mix of both above examples. Order matters !::
...
@@ -130,6 +130,16 @@ A mix of both above examples. Order matters !::
apachedex --skip-base "/site1/ignored(/|$|\?)"
apachedex --skip-base "/site1/ignored(/|$|\?)"
--base "/site1(/|$|\?)" "/site2(/|$|\?)"
--base "/site1(/|$|\?)" "/site2(/|$|\?)"
Matching non-ASCII urls works by using urlencoded strings::
apachedex --base "/%E6%96%87%E5%AD%97%E5%8C%96%E3%81%91(/|$|\\?)" access.log
Naming websites so that report looks less intimidating, by interleaving
"+"-prefixed titles with regexes (title must be just before regex)::
apachedex --default "Public website" --base "+Back office"
"/backoffice(/|$|\\?)" "+User access" "/secure(/|$|\\?)" access.log
Saving the result of an analysis for faster reuse::
Saving the result of an analysis for faster reuse::
apachedex --default foo --format json --out save_state.json --period day
apachedex --default foo --format json --out save_state.json --period day
...
@@ -161,6 +171,14 @@ For better performance...
...
@@ -161,6 +171,14 @@ For better performance...
bzcat access.log.bz2 | apachedex [...] -
bzcat access.log.bz2 | apachedex [...] -
- when letting apachedex decide statistic granularity with multiple log files,
provide earliest and latest log files first (whatever order) so apachedex can
adapt its data structure to analysed time range before there is too much
data::
apachedex [...] access.log.1.gz access.log.99.gz access.log.2.gz
access.log.3.gz [...] access.98.gz
- parse log files in parallel processes, saving analysis output and aggregating
- parse log files in parallel processes, saving analysis output and aggregating
them in the end::
them in the end::
...
...
apachedex/__init__.py
View file @
a4c6f5d7
...
@@ -1076,11 +1076,7 @@ def main():
...
@@ -1076,11 +1076,7 @@ def main():
help
=
'Suppress progress indication (file being parsed, lines counter). '
help
=
'Suppress progress indication (file being parsed, lines counter). '
'Does not imply -q.'
)
'Does not imply -q.'
)
parser
.
add_argument
(
'--state-file'
,
nargs
=
'+'
,
default
=
[],
parser
.
add_argument
(
'--state-file'
,
nargs
=
'+'
,
default
=
[],
help
=
'Use given JSON files as initial state. Use - for stdin. Loading'
help
=
'Use given JSON files as initial state. Use - for stdin.'
)
'multiple files through stdin is not possible). Mixing '
'files generated with different parameters is allowed, but no '
'correction is made. Output may be unusable (ex: different --apdex, '
'different --period, ...).'
)
group
=
parser
.
add_argument_group
(
'generated content (all formats)'
)
group
=
parser
.
add_argument_group
(
'generated content (all formats)'
)
group
.
add_argument
(
'-a'
,
'--apdex'
,
default
=
1.0
,
type
=
float
,
group
.
add_argument
(
'-a'
,
'--apdex'
,
default
=
1.0
,
type
=
float
,
...
@@ -1091,11 +1087,7 @@ def main():
...
@@ -1091,11 +1087,7 @@ def main():
group
.
add_argument
(
'-f'
,
'--format'
,
choices
=
format_generator
,
group
.
add_argument
(
'-f'
,
'--format'
,
choices
=
format_generator
,
default
=
'html'
,
help
=
'Format in which output should be generated.'
)
default
=
'html'
,
help
=
'Format in which output should be generated.'
)
group
.
add_argument
(
'-p'
,
'--period'
,
choices
=
period_parser
,
group
.
add_argument
(
'-p'
,
'--period'
,
choices
=
period_parser
,
help
=
'Periodicity of sampling buckets. Default: (decide from data). '
help
=
'Periodicity of sampling buckets. Default: (decide from data).'
)
'Performance note: leaving out this parameter reduces parsing '
'performance, as each period increase requires re-dispatching already '
'processed data. To mitigate this, provide earliest and latest log '
'files before all others (ex: log0 log3 log1 log2).'
)
group
=
parser
.
add_argument_group
(
'generated content (html)'
)
group
=
parser
.
add_argument_group
(
'generated content (html)'
)
group
.
add_argument
(
'-s'
,
'--stats'
,
action
=
'store_true'
,
group
.
add_argument
(
'-s'
,
'--stats'
,
action
=
'store_true'
,
...
@@ -1114,25 +1106,18 @@ def main():
...
@@ -1114,25 +1106,18 @@ def main():
help
=
'apdex graph ordinate scale. Default: %(default)s'
)
help
=
'apdex graph ordinate scale. Default: %(default)s'
)
group
=
parser
.
add_argument_group
(
'site matching'
,
'Earlier arguments take '
group
=
parser
.
add_argument_group
(
'site matching'
,
'Earlier arguments take '
'precedence. For example: --skip-base "/foo/bar(/|$|
\
\
?)" '
'precedence. Arguments are Python regexes, matching urlencoded strings.'
'--base "/foo(/|$|
\
\
?)" generates stats for /foo, excluding /foo/bar. '
'Regex matches can be named by providing a "+"-prefixed string before '
'Arguments (except for -d/--default) are interpreted as Python regexes. '
'regex.'
)
'Literal values are expected urlencoded. For example: '
'--base "/%E6%96%87%E5%AD%97%E5%8C%96%E3%81%91(/|$|
\
\
?)" matches '
'"/
\
xe6
\
x96
\
x87
\
xe5
\
xad
\
x97
\
xe5
\
x8c
\
x96
\
xe3
\
x81
\
x91
" ("mojibake").'
'You can name matched entries by providing a name before the regex,'
'prefixed by "+". For example: --base +foo "/foo(/|$|
\
\
?)" '
'"/bar(/|$|
\
\
?)" "+baz boo" "/baz(/|$|
\
\
?)" will defines 3 bases, named'
'"foo", "/bar(/|$|
\
\
?)" and "baz boo" respectively.'
)
group
.
add_argument
(
'-d'
,
'--default'
,
group
.
add_argument
(
'-d'
,
'--default'
,
help
=
'Caption for lines matching no prefix, or skip them if not provided.'
)
help
=
'Caption for lines matching no prefix, or skip them if not provided.'
)
group
.
add_argument
(
'--base'
,
dest
=
'path'
,
default
=
([],
{}),
nargs
=
'+'
,
group
.
add_argument
(
'--base'
,
dest
=
'path'
,
default
=
([],
{}),
nargs
=
'+'
,
action
=
AggregateSiteUrl
,
action
=
AggregateSiteUrl
,
help
=
'
Absolute base url(s) of some part
of a site.'
)
help
=
'
Title (optional) and regexes matching parts
of a site.'
)
group
.
add_argument
(
'--erp5-base'
,
dest
=
'path'
,
nargs
=
'+'
,
group
.
add_argument
(
'--erp5-base'
,
dest
=
'path'
,
nargs
=
'+'
,
action
=
AggregateSiteUrl
,
action
=
AggregateSiteUrl
,
help
=
'
Absolute base url(s) of some part of an ERP5 site (more
'
help
=
'
Similar to --base, but with specialised statistics. Ex:
'
'
specific stats than --base).
'
)
'
"/erp5(/|$|
\
?)
"
'
)
group
.
add_argument
(
'--skip-base'
,
dest
=
'path'
,
nargs
=
'+'
,
group
.
add_argument
(
'--skip-base'
,
dest
=
'path'
,
nargs
=
'+'
,
action
=
AggregateSiteUrl
,
action
=
AggregateSiteUrl
,
help
=
'Absolute base url(s) to ignore.'
)
help
=
'Absolute base url(s) to ignore.'
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment