Python Forum
Extracting Specific Lines from text file based on content.
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Extracting Specific Lines from text file based on content.
#1
Hey, all. My apologies as I'm new to python in general, so I'm doing a lot of reading that goes over my head. :)

I'm working on a multi-step process where I run one python script that makes an API call to a system. I then have to extract some of those values from that output so that I can submit them, along with a new password, via a second API call that I would make after.

I have two main questions, first, is it possible to pipe the output of the initial API call directly to a text file rather than just to the console?

Second, is it even possible for python to look through a text file and extract certain lines based on the values in that line?

Some sample output from the first API call:
Output:
{ "LINE_1": true, "LINE_2": "Post \"https://a:443/sdk\": dial tcp: lookup a on 1.1.127.252:53: no such host", "EXAMPLE_3": "", "created_at": 1648040001, "deleted": false, "deleted_at": 0, "delta_interval": 60, "description": "API's Testing.", "disable_backup": false, "hosts_list": [ { "host_name": "TestingHost", "port_number": 443 } ], "id": "623b1841755f021ebe269xyz", "name": "Tester", "password": "", "scope_id": "5f7e2a15497d4f48a9605687", "type": "vcenter", }
There are four specific lines I am trying to extract:
"name":
"type":
"hosts_list":
"username":

I've come across some things describing how to convert lines of a text file into individual objects, but that only allows me to print lines based on the number line they are in a text file, it doesn't print lines based on the content in their lines.

Also, if this is possible, the "hosts_list" line may cause an issue because it actually has a subsection that can contain multiple entries. In the above example the subsection is:
Output:
{ "host_name": "TestingHost", "port_number": 443 }
Thank you for any help you can provide.
Reply
#2
You don't even have to pipe it to a text file. You can read it as input. Can you show me how you are making the API calls?
Reply
#3
(Mar-25-2022, 06:17 PM)deanhystad Wrote: You don't even have to pipe it to a text file. You can read it as input. Can you show me how you are making the API calls?

I'm not sure what you mean "read it as input," any chance you can explain?

My API call is:

API_ENDPOINT="https://MyURL.com"

restclient = RestClient(API_ENDPOINT,
                credentials_file='MyCredentialsFile.json',
                verify=False)

resp = restclient.get('API_Call_URI_here')
Reply
#4
You did not specify you were writing a RESTful client. API is a really common term and I don't do a lot of web stuff. The json makes more sense now.

resp = restclient.get('API_Call_URI_here') returns a response, it does not write to the console. If something is being written to the console it is not because of the API call. How are you running this code? If you are running Python from the interactive window, the REPL automatically prints a reply to each command. For your command it would print resp.json. This is an artifact of how you are running the code and not real output.

resp will hold the response to your "get". resp.status_code is the status code for the request. resp.json is a json string with all the interesting stuff. You can convert the json string to a Python dictionary to make it easier to work with.
import requests
import json

API_ENDPOINT="https://MyURL.com"
 
restclient = requests.RestClient(API_ENDPOINT,
                credentials_file='MyCredentialsFile.json',
                verify=False)
 
resp = restclient.get('API_Call_URI_here')

data = json.loads(resp.json)
print(data["hosts_list"])
This should print:
Output:
[ { "host_name": "TestingHost", "port_number": 443 } ]
Using the dictionary you can ask for any of the json fields using their key name. These are the keys in the response you posted.
Quote:"LINE_1", "LINE_2", "EXAMPLE_3", "created_at", "deleted", "deleted_at", "delta_interval",
"description", "disable_backup"' "hosts_list", "id", "name", "password"' "scope_id", "type"
Are you familiar with Python dictionaries?
Reply
#5
deanhystad Wrote:resp.json is a json string with all the interesting stuff. You can convert the json string to a Python dictionary to make it easier to work with.
deanhystad as info so do Requests have build in json encoder/decoder added a long time ago(2014).
So the extra step with json string trough json library is not recommend.
can also mess up encoding,resp.json() directly from Request dos a much better job of figuring out encoding used.
>>> import requests
>>> 
>>> resp = requests.get('https://github.com/timeline.json')
>>> resp.json()
{'documentation_url': 'https://docs.github.com/v3/activity/events/#list-public-events',
 'message': 'Hello there, wayfaring stranger. If you’re reading this then you '
            'probably didn’t see our blog post a couple of years back '
            'announcing that this API would go away: http://git.io/17AROg Fear '
            'not, you should be able to get what you need from the shiny new '
            'Events API instead.'}

>>> resp.json()['documentation_url']
'https://docs.github.com/v3/activity/events/#list-public-events'

>>> resp.encoding
'utf-8'
Reply
#6
That makes sense since it is something you would have to do all the time. Can you provide a link to a nice tutorial that demonstrates actually doing something with the json reply? I don't do this kind of thing, but it looks interesting.
Reply
#7
(Mar-25-2022, 09:55 PM)deanhystad Wrote: Can you provide a link to a nice tutorial that demonstrates actually doing something with the json reply?
In this post get Tor Mag to work that dos not have puplic API.
First send some post Payload then parse r.json() to get the shortened URL.

In this Thread see use of httpbin,can to testing of HTTP Request.

If search so are many tutorials about this eg this one is ok.
How to Use the Python Requests Module With REST APIs
Reply
#8
Thanks for this. When I run the script with
resp.json()
I get no output, but when I run it with
print (json.dumps(resp.json(), indent=2))
I get the full output. When I try to add in specific libraries (such as "hosts_file") I get the error
Error:
resp.json()["hosts_file"]
.

Is there something else I need to add?

======= EDIT =======

I needed to add the indexing piece (not sure correct terminology), but it is the [0] part in the below code.

print (resp.json()[0]['id'])
(Mar-25-2022, 09:41 PM)snippsat Wrote:
deanhystad Wrote:resp.json is a json string with all the interesting stuff. You can convert the json string to a Python dictionary to make it easier to work with.
deanhystad as info so do Requests have build in json encoder/decoder added a long time ago(2014).
So the extra step with json string trough json library is not recommend.
can also mess up encoding,resp.json() directly from Request dos a much better job of figuring out encoding used.
>>> import requests
>>> 
>>> resp = requests.get('https://github.com/timeline.json')
>>> resp.json()
{'documentation_url': 'https://docs.github.com/v3/activity/events/#list-public-events',
 'message': 'Hello there, wayfaring stranger. If you’re reading this then you '
            'probably didn’t see our blog post a couple of years back '
            'announcing that this API would go away: http://git.io/17AROg Fear '
            'not, you should be able to get what you need from the shiny new '
            'Events API instead.'}

>>> resp.json()['documentation_url']
'https://docs.github.com/v3/activity/events/#list-public-events'

>>> resp.encoding
'utf-8'
Reply
#9
What library as using,there is no RestClient in Requests?
Is the old python-restclient,which link to cheeseshop 🧀,
and use easy_install this is so old that new people to Python don't know what this is.

What showed in this Thread is Requests,and that's what you should use.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Extracting the correct data from a CSV file S2G 6 506 Jun-03-2024, 04:50 PM
Last Post: snippsat
  Copy Paste excel files based on the first letters of the file name Viento 2 673 Feb-07-2024, 12:24 PM
Last Post: Viento
  Extracting specific file from an archive tester_V 4 761 Jan-29-2024, 06:41 PM
Last Post: tester_V
  Color a table cell based on specific text Creepy 11 2,694 Jul-27-2023, 02:48 PM
Last Post: deanhystad
  Split pdf in pypdf based upon file regex standenman 1 2,417 Feb-03-2023, 12:01 PM
Last Post: SpongeB0B
Thumbs Up Need to compare the Excel file name with a directory text file. veeran1991 1 1,320 Dec-15-2022, 04:32 PM
Last Post: Larz60+
  Reading Specific Rows In a CSV File finndude 3 1,165 Dec-13-2022, 03:19 PM
Last Post: finndude
  How to remove footer from PDF when extracting to text jh67 3 5,893 Dec-13-2022, 06:52 AM
Last Post: DPaul
  reading content between a specific xml tag saisankalpj 1 985 Oct-31-2022, 01:37 PM
Last Post: wavic
  How to extract specific data from .SRC (note pad file) Shinny_Shin 2 1,437 Jul-27-2022, 12:31 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020