Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
F
fluent-plugin-wendelin
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Eteri
fluent-plugin-wendelin
Commits
f1c7bcc4
Commit
f1c7bcc4
authored
Apr 20, 2018
by
Eteri
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
solve different tag problem
parent
a8b179a1
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
84 additions
and
88 deletions
+84
-88
lib/fluent/plugin/wendelin_client.rb
lib/fluent/plugin/wendelin_client.rb
+84
-88
No files found.
lib/fluent/plugin/wendelin_client.rb
View file @
f1c7bcc4
...
@@ -15,99 +15,95 @@
...
@@ -15,99 +15,95 @@
# See the License for the specific language governing permissions and
# See the License for the specific language governing permissions and
# limitations under the License.
# limitations under the License.
require
'net/http'
require
'net/http'
require
'openssl'
require
'openssl'
require
'http'
# class representing a Wendelin client
# class representing a Wendelin client
class
WendelinClient
class
WendelinClient
# `streamtool_uri` - URI pointing to portal_input_data_stream "mountpoint"
# `streamtool_uri` - URI pointing to portal_input_data_stream "mountpoint"
# `credentials` # {'user' => _, 'password' => _} TODO change to certificate
# `credentials` # {'user' => _, 'password' => _} TODO change to certificate
# `log` - logger to use
# `log` - logger to use
def
initialize
(
streamtool_uri
,
credentials
,
log
)
def
initialize
(
streamtool_uri
,
credentials
,
log
)
@streamtool_uri
=
streamtool_uri
@streamtool_uri
=
streamtool_uri
@credentials
=
credentials
@credentials
=
credentials
@log
=
log
@log
=
log
end
end
# start request in an independent function to keep the connection open
# start request in an independent function to keep the connection open
def
start_request
(
uri
)
def
start_request
(
uri
)
puts
' START NEW REQUEST'
@http
=
Net
::
HTTP
.
start
(
uri
.
hostname
,
uri
.
port
,
:use_ssl
=>
(
uri
.
scheme
==
'https'
),
@http
=
Net
::
HTTP
.
start
(
uri
.
hostname
,
uri
.
port
,
:verify_mode
=>
OpenSSL
::
SSL
::
VERIFY_NONE
,
use_ssl:
(
uri
.
scheme
==
'https'
),
verify_mode:
OpenSSL
::
SSL
::
VERIFY_NONE
,
# Net::HTTP default open timeout is infinity, which results
# in thread hang forever if other side does not fully
# Net::HTTP default open timeout is infinity, which results
# establish connection. Default read_timeout is 60 seconds.
# in thread hang forever if other side does not fully
# We go safe way and make sure all timeouts are defined.
# establish connection. Default read_timeout is 60 seconds.
:ssl_timeout
=>
60
,
# We go safe way and make sure all timeouts are defined.
:open_timeout
=>
60
,
ssl_timeout:
60
,
:read_timeout
=>
60
,
open_timeout:
60
,
:keep_alive_timeout
=>
60
,)
read_timeout:
60
,
end
keep_alive_timeout:
60
)
end
# ingest `data_chunk` to a stream referenced as `reference`
def
ingest
(
reference
,
data_chunk
)
# ingest `data_chunk` to a stream referenced as `reference`
uri
=
URI
(
"
#{
@streamtool_uri
}
/ingest?reference=
#{
reference
}
"
)
def
ingest
(
reference
,
data_chunk
)
# call start_request if request is undefined
uri
=
URI
(
"
#{
@streamtool_uri
}
/ingest?reference=
#{
reference
}
"
)
@request
||=
start_request
(
uri
)
puts
'uri = '
@request
=
Net
::
HTTP
::
Post
.
new
(
uri
)
puts
uri
# call start_request if request is undefined
# connect again if the connection is not started
@request
||=
start_request
(
uri
)
if
!
@http
.
started?
()
start_request
(
uri
)
# connect again if the connection is not started
end
start_request
(
uri
)
unless
@http
.
started?
# When using 'application/x-www-form-urlencoded', Ruby encodes with regex
@request
=
Net
::
HTTP
::
Post
.
new
(
uri
)
# and it is far too slow. Such POST is legit:
# https://stackoverflow.com/a/14710450
# When using 'application/x-www-form-urlencoded', Ruby encodes with regex
@request
.
body
=
data_chunk
# and it is far too slow. Such POST is legit:
@request
.
content_type
=
'application/octet-stream'
# https://stackoverflow.com/a/14710450
@request
.
body
=
data_chunk
if
@credentials
.
has_key?
(
'user'
)
@request
.
content_type
=
'application/octet-stream'
@request
.
basic_auth
@credentials
[
'user'
],
@credentials
[
'password'
]
end
if
@credentials
.
key?
(
'user'
)
@request
.
basic_auth
@credentials
[
'user'
],
@credentials
[
'password'
]
@log
.
on_trace
do
end
@log
.
trace
'>>> REQUEST'
@log
.
trace
"method
\t
=>
#{
@request
.
method
}
"
@log
.
on_trace
do
@log
.
trace
"path
\t
=>
#{
@request
.
path
}
"
@log
.
trace
'>>> REQUEST'
@log
.
trace
"uri
\t
=>
#{
@request
.
uri
}
"
@log
.
trace
"method
\t
=>
#{
@request
.
method
}
"
@log
.
trace
"body
\t
=>
#{
@request
.
body
}
"
@log
.
trace
"path
\t
=>
#{
@request
.
path
}
"
@log
.
trace
"body_stream
\t
=>
#{
@request
.
body_stream
}
"
@log
.
trace
"uri
\t
=>
#{
@request
.
uri
}
"
@request
.
each
{
|
h
|
@log
.
trace
"
#{
h
}
:
\t
#{
@request
[
h
]
}
"
}
@log
.
trace
"body
\t
=>
#{
@request
.
body
}
"
@log
.
trace
@log
.
trace
"body_stream
\t
=>
#{
@request
.
body_stream
}
"
end
@request
.
each
{
|
h
|
@log
.
trace
"
#{
h
}
:
\t
#{
@request
[
h
]
}
"
}
@log
.
trace
begin
end
res
=
@http
.
request
(
@request
)
# Net::HTTPResponse object
end
begin
res
=
@http
.
request
(
@request
)
# Net::HTTPResponse object
rescue
end
# some http/ssl/other connection error
rescue
StandardError
@log
.
warn
"HTTP ERROR:"
# some http/ssl/other connection error
raise
@log
.
warn
'HTTP ERROR:'
else
raise
@log
.
on_trace
do
else
@log
.
trace
'>>> RESPONSE'
@log
.
on_trace
do
res
.
each
{
|
h
|
@log
.
trace
"
#{
h
}
:
\t
#{
res
[
h
]
}
"
}
@log
.
trace
'>>> RESPONSE'
@log
.
trace
"code
\t
=>
#{
res
.
code
}
"
res
.
each
{
|
h
|
@log
.
trace
"
#{
h
}
:
\t
#{
res
[
h
]
}
"
}
@log
.
trace
"msg
\t
=>
#{
res
.
message
}
"
@log
.
trace
"code
\t
=>
#{
res
.
code
}
"
@log
.
trace
"class
\t
=>
#{
res
.
class
}
"
@log
.
trace
"msg
\t
=>
#{
res
.
message
}
"
@log
.
trace
"body:"
,
res
.
body
@log
.
trace
"class
\t
=>
#{
res
.
class
}
"
end
@log
.
trace
'body:'
,
res
.
body
end
if
res
.
kind_of?
(
Net
::
HTTPSuccess
)
# res.code is 2XX
#@log.info "ingested ok"
if
res
.
is_a?
(
Net
::
HTTPSuccess
)
# res.code is 2XX
else
# @log.info "ingested ok"
@log
.
warn
"FAIL:"
else
res
.
value
@log
.
warn
'FAIL:'
end
res
.
value
end
end
end
end
end
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment