Friday 22 August 2014
Http Source in Flume
A Http source accepts Flume Events by HTTP POST and GET. GET should be used for experimentation only. HTTP requests are converted into flume events by a pluggable "handler" which must implement the HTTPSourceHandler interface. This handler takes a HttpServletRequest and returns a list of flume events.
handler: the class that deserializes a HttpServletRequest into a list of flume events. This class must implement HTTPSourceHandler. Default: JSONHandler.
A JSON handler which converts JSON objects to Flume events is provided.
Configure Http Source:
httpagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
httpagent.sources.http-source.bind = 127.0.0.1
httpagent.sources.http-source.port = 8080
httpagent.sources.http-source.channels=memory-channel
Test:
$flume-ng agent -f json_agent.conf -n json_agent
$curl -X POST -H 'Content-Type: application/json; charset=UTF-8' -d '[{"username":"xyz","password":"123"}]' http://127.0.0.1:8080
Error:
Everything looks fine. But don't know why the written file is empty in hdfs.
We need to an Serializer by implementing the EventSerializer interface and the write() method,
An example is below.
https://github.com/wdavidw/flume
Reference:
http://flume.apache.org/releases/content/1.4.0/apidocs/org/apache/flume/source/http/HTTPSource.html
Labels:
Flume
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment