IT

Grep IP address from log file: detailed how-to

View


You need to find out who is accessing the system.This often means what you have to do grep IP address of the log file. Grep is a command line tool for searching text in files using. Regular expression syntax.

Let’s see how to use grep to look up an IP address in a log file and how to use a regular expression to look up an address in different situations.

This tutorial uses a sample HTTPD access log.You can follow by downloading the file and opening it Link In another tab, save the page as a file on your computer.

Grep with the exact IP address in the log file

First, let’s see how to find the exact address in the access log.

Run grep with the IP address you are looking for and the name of the log file.

$ grep 46.72.177.4 access.log 
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [14/Dec/2015:16:39:27 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [14/Dec/2015:16:39:28 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" “-“~
188.187.105.165 - - [30/Jan/2019:10:57:26 +0100] "GET /apache-log/46.72.177.4%20-%20-%20[12/Dec/2015:18:31:08%20+0100]%20%22GET%20/administrator/%20HTTP/1.1%22%20200%204263%20%22-%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 417 "-" "Wget/1.20.1 (darwin17.7.0)" "-"
188.187.105.165 - - [30/Jan/2019:10:57:26 +0100] "GET /apache-log/46.72.177.4%20-%20-%20[12/Dec/2015:18:31:08%20+0100]%20%22POST%20/administrator/index.php%20HTTP/1.1%22%20200%204494%20%22http://almhuette-raith.at/administrator/%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 466 "-" "Wget/1.20.1 (darwin17.7.0)" "-"
188.187.105.165 - - [30/Jan/2019:10:59:44 +0100] "GET /apache-log/46.72.177.4%20-%20-%20[14/Dec/2015:16:39:27%20+0100]%20%22GET%20/administrator/%20HTTP/1.1%22%20200%204263%20%22-%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 417 "-" "Wget/1.20.1 (darwin17.7.0)" "-"
188.187.105.165 - - [30/Jan/2019:10:59:44 +0100] "GET /apache-log/46.72.177.4%20-%20-%20[14/Dec/2015:16:39:28%20+0100]%20%22POST%20/administrator/index.php%20HTTP/1.1%22%20200%204494%20%22http://almhuette-raith.at/administrator/%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 466 "-" "Wget/1.20.1 (darwin17.7.0)" "-"

I cropped the results from the sample file to save space. Do this for most results.

Grep ran as you requested.Found all instances of 46.72.177.4 Returns the line that is in the file and contains it. Some lines start with the IP address you are looking for. This shows the HTTP request from that address. Others include a reference to it.

What if I only want to see the request?Try pasting with a regular expression that matches the line that starts with 46.72.177.4..

$ grep "^46.72.177.4" access.log 
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [14/Dec/2015:16:39:27 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [14/Dec/2015:16:39:28 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [15/Dec/2015:18:16:52 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [15/Dec/2015:18:16:52 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [17/Dec/2015:19:43:47 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [17/Dec/2015:19:43:47 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"

The result set is much smaller because * ^ * means it matches the next string only if it is at the beginning of a line.

Therefore, a specific IP address will be displayed each time you access this server. If you are looking for a count, pipe the results through the word counting utility * wc *.

$ grep "^46.72.177.4" access.log | wc -l
8

The sample IP address made 8 requests.

It covers many areas. But what if you need to match part of the address?

Grep some of the IP addresses in the log file

Instead of searching for a specific IP address, we recommend that you search for part of the address.

Let’s search for the first two octets of the address, the Class B network. We are looking for this address both as a requester and in the request.

$ grep "46.72" access.log |more
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34
.0) Gecko/20100101 Firefox/34.0" "-"
46.72.213.133 - - [12/Dec/2015:18:39:27 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.213.133 - - [12/Dec/2015:18:39:27 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:
34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.184.174 - - [12/Dec/2015:18:51:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.184.174 - - [12/Dec/2015:18:51:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:
34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.185.236 - - [12/Dec/2015:19:31:11 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.185.236 - - [12/Dec/2015:19:31:12 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:
34.0) Gecko/20100101 Firefox/34.0" "-"
37.159.185.154 - - [27/Aug/2020:15:48:21 +0200] "GET /apache-log/access.log HTTP/1.1" 200 14637264 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36" "-"
3.121.24.234 - - [27/Aug/2020:19:14:42 +0200] "GET /apache-log/access.log HTTP/1.1" 200 16846272 "-" "Mozilla/5.0 (Linux; Android 6.0; Nexus 7 Build/KRT16M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36" "-"
3.121.24.234 - - [29/Aug/2020:03:44:42 +0200] "GET /apache-log/access.log HTTP/1.1" 200 10446872 "-" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Mobile Safari/537.36" "-"
172.58.204.254 - - [30/Aug/2020:01:08:17 +0200] "GET /apache-log/access.log HTTP/1.1" 200 1346272 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36" "-"

Grep matched the line I wasn’t looking for. why?

In regular expression syntax, the period matches “any character”. So you get a row that contains 46872 and 46272, among other things.

Let’s move to Extended regular expression..

$grep -E "46.72" access.log |more
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34
.0) Gecko/20100101 Firefox/34.0" "-"
46.72.213.133 - - [12/Dec/2015:18:39:27 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.213.133 - - [12/Dec/2015:18:39:27 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:
34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.184.174 - - [12/Dec/2015:18:51:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.184.174 - - [12/Dec/2015:18:51:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:
34.0) Gecko/20100101 Firefox/34.0" "-"
188.187.105.165 - - [30/Jan/2019:11:03:10 +0100] "GET /apache-log/46.72.192.202%20-%20-%20[18/Dec/2015:07:54:10%20+0100]%20%22GET%20/administrator/%20HTTP/1.1%22%20200%204263%20%22-%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 419 "-" "Wget/1.20.1 (darwin17.7.0)" "-"
188.187.105.165 - - [30/Jan/2019:11:03:10 +0100] "GET /apache-log/46.72.192.202%20-%20-%20[18/Dec/2015:07:54:10%20+0100]%20%22POST%20/administrator/index.php%20HTTP/1.1%22%20200%204494%20%22http://almhuette-raith.at/administrator/%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 468 "-" "Wget/1.20.1 (darwin17.7.0)" "-"
54.185.146.72 - - [29/Nov/2019:15:02:14 +0100] "GET /apache-log/access.log HTTP/1.1" 200 64168 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36" "-"
54.185.146.72 - - [13/Jan/2020:13:33:42 +0100] "GET /apache-log/access.log HTTP/1.1" 200 42904 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36" "-"

When passing -E To run grep, enable extended regular expressions. This will escape IP address period with backslash. Tells grep to match the literal period, not the letter. Therefore, “46.72” is exactly what you are looking for.

Unfortunately, it turns out that “what you are looking for” is not exactly what you are looking for. The last few lines above contain addresses ending in 46.72.

When searching with ”^ 46.72” Receive requests that begin with these two octets. However, you will miss the rows that contain them as part of the query.

Regular expressions have the following concepts: Word boundaries.. If you specify a word, it matches the expression only if it does not overlap with other characters. Therefore, the last two lines above are filtered for the following reasons: 46.72 The number 1 is prepended.

add to w For command line arguments to grep.

$ grep -Ew "46.72" access.log |more
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34
.0) Gecko/20100101 Firefox/34.0" "-"
46.72.213.133 - - [12/Dec/2015:18:39:27 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.213.133 - - [12/Dec/2015:18:39:27 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:
34.0) Gecko/20100101 Firefox/34.0" "-"
188.187.105.165 - - [30/Jan/2019:11:03:10 +0100] "GET /apache-log/46.72.192.202%20-%20-%20[18/Dec/2015:07:54:10%20+0100]%20%22GET%20/administrator/%20HTTP/1.1%22%20200%204263%20%22-%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 419 "-" "Wget/1.20.1 (darwin17.7.0)" "-"
188.187.105.165 - - [30/Jan/2019:11:03:10 +0100] "GET /apache-log/46.72.192.202%20-%20-%20[18/Dec/2015:07:54:10%20+0100]%20%22POST%20/administrator/index.php%20HTTP/1.1%22%20200%204494%20%22http://almhuette-raith.at/administrator/%22%20%22Mozilla/5.0%20(Windows%20NT%206.0;%20rv:34.0)%20Gecko/20100101%20Firefox/34.0%22%20%22-%22 HTTP/1.1" 404 468 "-" "Wget/1.20.1 (darwin17.7.0)" "-"

to add w The command flag indicates that only word matching is required. This ruled out unwanted matches. (If there is an address 46.72 However, it is displayed as the center or last octet. )

So far, you’ve only dealt with searching for addresses you know. What if I want to match any address?

Grep of any IP address in the log file

If you don’t know the address you’re looking for, you’ll need to create an expression that matches it. This requires a match between the character class and the wildcard.

A character class is a set of characters. The set is shown in square brackets. [ ].. The IP address consists of numbers.Therefore, we need a character class [0-9] Matches any number.

There are 4 sets of addresses Until Three numbers. Therefore, you must match at least one, but no more than, three numbers per octet. You can do this using curly braces. {}. [0-9]{1,3} Matches up to 3 numbers.

Therefore, the complete formula for the full IP address is: [0-9]{1,3} .[0-9]{1,3} .[0-9]{1,3} .[0-9]{1,3}

$ grep -E "[^^][0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}" access.log |more
109.169.248.247 - - [12/Dec/2015:18:25:11 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
109.169.248.247 - - [12/Dec/2015:18:25:11 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; r
v:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "GET /administrator/ HTTP/1.1" 200 4263 "-" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
46.72.177.4 - - [12/Dec/2015:18:31:08 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://almhuette-raith.at/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34
.0) Gecko/20100101 Firefox/34.0" "-"

This search matches all rows because we are using HTTP access.log.

What if I want to create a list of IP addresses without request information? Grep has a -o command line option that returns only the portion of each line that matches the regular expression.

$ grep -E -o "[^^][0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}" access.log |more 109.169.248.247 109.169.248.247 46.72.177.4 46.72.177.4

This will reduce noise and display a list of addresses. However, since this is an access log, there is a lot of duplication. It’s easy to fix.

$ grep -E -o "^[^^][0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}" access.log|uniq|more 109.169.248.247 46.72.177.4

The· uniq The command line tool filters duplicate items from the input. So if you pipe the output from grep, you’ll get a list of all the unique IP addresses in the file. You can also add -c to uniq to get the count for each address.

$ grep -E -o "^[^^][0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}" access.log|uniq -c|more 42 109.169.248.247 16 46.72.177.4

In this case, the two IP addresses appeared 42 and 16 times.

Grep and regular expressions

I used grep and regular expression syntax to find the IP address in the log file. We also added a uniq command to filter addresses into a list. These basic components provide everything you need for almost every situation.

It is better to analyze the IP address in the log file. Scalyr indexes logs in real time.So you can search for a specific address Power Queries. For example, you soon Display common IP addresses Find out which client is creating the most traffic. Therefore, instead of using command line tools to generate the report, the data you need is immediately available.

Sign up for a free trial Here And start super fast log analysis today!

This post was written by Eric Goebelbecker. Eric He has been working in the financial markets of New York City for 25 years, developing the infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about why teams are effective (or less effective).

https://prod-design.scalyr.com/blog/grep-ip-address-log-file/ Grep IP address from log file: detailed how-to

Back to top button