Last updated at Mon, 06 Nov 2017 20:29:30 GMT
We’re very excited to announce that our REST Query API is now available. With this API, you can:
- make it easy to remotely query your log data
- easily integrate Logentries with third party solutions, external systems and internal tools
- allow users and systems to query their log data programmatically over our REST API
In this article, I will show how you can quickly interact with the Query API by sending in a LEQL query and getting the results back – this might be useful if you want to easily integrate data or calculations from Logentries in your own dashboard or web app.
You can use any number of programming languages to do this – for the purposes of this article, I am using JQuery, and I’ve included some code snippets below.
This is a simple webpage that allows me to enter the log, time range and the query that I want to run.
The results will be a JSON object that contains either the matching log entries, or if I ran a calculation function, the statistics for the query.
First things first
Don’t have a Logentries account? Sign up for your free trial.
Getting your API key
- To get an API Key, browse to the account area of Logentries by clicking on the profile icon on the left hand navigation.
- Click the Api Keys tab.
- If you have not previously generated any API keys for your account, you can do so by clicking the “generate” link for the type of key you want to use. As we do not need update access to use the Query API, I used a Read Only key.
Getting your Log Key
You will need to tell the Query API which logs you want to query against; to do this, you will pass in the Log Key. Note: this is not the same as the token – the Log Key does not allow write access to the Log.
Browse to the Log you wish to query, and click on the “Settings” tab. Under the Log name field you will see the key. Make a note of the key.
Hitting the API
Forming and sending the request
So now we’re ready to start building the request. In addition to the Log Key and the API Key, you will also need to specify the start and end time, and of course the LEQL query that you want to run. The start and end time must be represented as epochs, so remember to convert the date format to this.
Here’s a breakdown of the URI that you will need to access:
https://rest.logentries.com/query/logs/<LOG KEY ID>?from=<START TIME>&to=<END TIME>&query=<LEQL QUERY>
So let’s say that my API Key is “123ABC987XYZ” and my Log Key is “L0G1DH3R3”. I want to search from 17:00 to 17-15 on the 14th April and I want to find all entries where a key name of “method” is equal to “GET”.
This is the equivalent of doing this in the Logentries UI:
That means my URI needs to be:
https://rest.logentries.com/query/logs/L0G1DH3R3?from=1460649600000&to=1460650500000&query=where(method%3DGET)
As you can see, the API Key is not included – I will send this as a header in the request instead. The first two lines are to convert the date into a timestamp.
toDate = new Date(toDate);
toDate = toDate.getTime();
$.ajax({
type: "GET",
headers: {"x-api-key" : "123ABC987XYZ"},
url: "https://rest.logentries.com/query/logs/L0G1DH3R3?from=1460649600000&to=1460650500000&query=where(method%3DGET)",
dataType: "json",
processData: false
})
Parsing the results
Can I see the results now? Not quite yet – as some queries may take a little longer to return results, the API will return a new URI to my results, rather than keep the connection open until the results are ready. When you hit the results URI, I will get back a “continue” URI until the results are ready. This reduces the risk of timeouts.
First, I need a handler to process the response from our first call. This function looks for the first node in the JSON with a name of “links” – this is the URI I need to hit in order to check for the results.
$.ajax({
type: "GET",
headers: {"x-api-key" : "123ABC987XYZ"},
url: fullURL,
data: sMessage,
dataType: datatype,
processData: false
})
.done(function(result)
{
myObj = result;
if (myObj["links"])
{
var queryURL = myObj["links"][0].href;
getResults(queryURL);
}
})
Now I can access the URI to get my results. Again, I need to pass in my API Key in the header. I’m going to do the subsequent call in a new function called getResults().
The response from this call will be a JSON object, and in this code snippet below, I am seeing if it contains a list of log entries or a list of statistics. As I didn’t run a LEQL query with a calculation, I expect to get back a list of entries. The response will look like this:
{
"logs": [
"L0G1DH3R3"
],
"events": [
{
"timestamp": 1460649601685,
"message": "2016-04-14 15:22:51 at=info method=GET path=/images/dell.gif host=aaa.herokuapp.com fwd=\"=400/NX\" dyno=web.1 connect=1ms service=957ms status=400 bytes=7456"
},
{
"timestamp": 1460649603682,
"message": "2016-04-14 15:22:51 at=info method=GET path=/images/demo.gif host=aaa.herokuapp.com fwd=\"215.22.3.226/NX\" dyno=web.1 connect=1ms service=967ms status=200 bytes=2368"
},
{
"timestamp": 1460649607695,
"message": "2016-04-14 15:23:02 at=info method=GET path=/images/dell.gif host=aaa.herokuapp.com fwd=\"=400/NX\" dyno=web.1 connect=1ms service=393ms status=400 bytes=7456"
},
snipped....
],
"links": [
{
"rel": "Next",
"href": "https://rest.logentries.com/query/logs/L0G1DH3R3?query=where(method%3DGET)&from=1460649814984&to=1460650500000&per_page=50&sequence_number=2029145718"
}
],
"leql": {
"during": {
"from": 1460649600000,
"to": 1460650500000
},
"statement": "where(method=GET)"
}
}
Now to just parse the list of events and print them to a text box on my page:
$.ajax({
type: "GET",
headers: {"x-api-key" : apikey},
url: fullURL,
data: sMessage,
dataType: datatype,
processData: false
})
.done(function(result)
{
myEvents = result;
var txtOut = "";
if (myEvents.events)
{
for (i = 0; i < myEvents.events.length; i++)
{
var line = myEvents.events[i].message + "\n";
txtOut+= line;
}
$("#txaOutput").val(txtOut);
}
else if(myEvents.statistics)
{
$("#txaOutput").val(JSON.stringify(myEvents, null, 2));
}
})
If I change my query to do a calculation – like where(method=GET) calculate(count), then the code above is going to just print out the raw JSON into a text box. That’s not too user friendly, so I want to render the results in a table instead.
This is the JSON I get back from my calculate query:
{
"statistics": {
"cardinality": 0,
"granularity": 90000,
"count": 0,
"from": 1460649600000,
"to": 1460650500000,
"stats": {
"global_timeseries": {
"count": 236
}
},
"groups": [],
"status": 200,
"timeseries": {
"global_timeseries": [
{
"count": 6
},
{
"count": 29
},
{
"count": 32
},
{
"count": 32
},
{
"count": 22
},
{
"count": 27
},
{
"count": 20
},
{
"count": 25
},
{
"count": 19
},
{
"count": 24
}
]
}
},
"leql": {
"during": {
"from": null,
"to": null
},
"statement": null
}
}
Let’s compare it to the same query when I run it in the UI:
The results match up as expected, but there’s no start and end date for each time slice. I want my page to look a bit more like the Logentries UI, and I can do this easily by looping through the results and adding the “granularity” key from the response to the “from” timestamp. While doing this, I’ll add the formatted results to a table.
var startDate = myEvents.statistics.from;
var timeDiff = myEvents.statistics.granularity;
var runningDiff = 0;
$("#statTable").append("<tr><th>Date Time</th><th>Result</th></tr>");
for (i = 0; i < myEvents.statistics.timeseries.global_timeseries.length; i++)
{
var sliceStartDate = startDate + runningDiff;
runningDiff += timeDiff;
var td1 = $("<td nowrap=nowrap>").html(new Date(sliceStartDate));
var td2 = $("<td>").html(myEvents.statistics.timeseries.global_timeseries[i].count);
var newRow = $("<tr>").append(td1).append(td2);
$("#statTable").append(newRow);
}
$("#statTable").append("<tr><th>Total</th><th>" + myEvents.statistics.stats.global_timeseries.count + "</th></tr>");
If you run a non-count calculation (e.g. average, max, percentile etc), then the JSON structure will be slightly different.
for (i = 0; i < myEvents["statistics"]["timeseries"][keyname].length; i++)
{
var sliceStartDate = startDate + runningDiff;
runningDiff += timeDiff;
var td1 = $("").html(new Date(sliceStartDate));
var td2 = $("
").html(myEvents["statistics"]["timeseries"][keyname][i][calculateFn]);
var newRow = $("
").append(td1).append(td2);
$("#statTable").append(newRow);
}
Recap
The sequence is:
- Build your query with a start and end date, the LEQL Query you want to run and the Log Key
- Make a request to the REST API URI, passing in your API Key in the header of the request
- Get back a response containing the URI to your results
- Poll that URI until it returns your results, as complex queries or queries spanning a large set of entries may take longer to complete
- Parse the results and display in your own application
Some questions you may have
How do I query multiple logs?
The method for this is slightly different – you will need to post a JSON object that contains the query, dates and Logs’ Keys to the REST API. Read more in our docs.
Will this return huge payloads?
No, the results are divided into pages of 50 entries – you will need to iterate through the pages. See the “next” link in the JSON events response above.
Should I use this in production?
As this example is entirely client side script for demonstration purposes, the API and Log Keys are very easily visible to anyone using the page. I’d recommend that the calls to the API are done by server-side code.
