Find a specific keyword after another keyword and change the output

[sgtmcc]

I will start off by saying I am new to Python and I have searched Google for a fix to my problem. Maybe I did not search it well enough, or did not search it the right way, but I have not found an answer that is specific to my scenario.

I have a text file which reads:

"group_id":"1","severity":"low","status":not_a_finding
"group_id":"2","severity":"low","status":not_a_finding
"group_id":"3","severity":"low","status":not_a_finding

etc etc

I have tried to just narrow it down to searching for only the status on the line of group_id 1, but I can't even narrow that down correctly

What I am trying to do is open the file, find a certain group id, and change the status to something else. All my searches just show how to search for it with "re"
Again, this is probably simple and I just am not searching correctly, but I'm not sure how else to go about it.

[menator01]

What is the code you have tried?

[sgtmcc]

Guess I should have included that, my mistake.

import re
f = open('test.txt', 'r')
content = f.read()
keyword = "1"
re.findall(keyword ,content)
re.search(keyword ,content)

findall outputs: ['1']
search outputs:<re.Match object; span=(12, 13), match='1'>

As I said earlier, I am just starting with narrowing down the one status. I was searching by keyword "1" to eliminate the other 2 group_id numbers.

[deanhystad]

You don't need to use regular expression pattern matching for this. If you want group 1, filter out lines that don't start with "group_id":"1".

The data is almost structured like a json file. Is the format of your text file accurately represented or did you leave some things out? Each line looks like a dictionary, but "not_a_finding" isn't something that would be recognized by the json parser.

I would be tempted to turn the lines back into dictionaries.

import pandas as pd


data = []
with open("test.txt", "r") as file:
    for line in file:
        if line := line.strip():
            # Remove all the quote characters from the string and
            # split into items
            items = line.replace('"', "").split(",")
            # Convert the key/value pairs into a dictionary.
            data.append(dict(field.split(":") for field in items))
print(*data, sep="\n")

Output:
{'group_id': '1', 'severity': 'low', 'status': 'not_a_finding'}
{'group_id': '2', 'severity': 'low', 'status': 'not_a_finding'}
{'group_id': '3', 'severity': 'low', 'status': 'not_a_finding'}
{'group_id': '1', 'severity': 'med', 'status': 'not_a_finding'}
{'group_id': '2', 'severity': 'med', 'status': 'not_a_finding'}
{'group_id': '3', 'severity': 'med', 'status': 'not_a_finding'}
{'group_id': '1', 'severity': 'high', 'status': 'not_a_finding'}
{'group_id': '2', 'severity': 'high', 'status': 'not_a_finding'}
{'group_id': '3', 'severity': 'high', 'status': 'not_a_finding'}

I added some lines to the file. Now I have a list of dictionaries. You can do interesting things with a list of dictionaries, like filtering.

print("Group 1", *[entry for entry in data if entry["group_id"] == "1"], "\n", sep="\n")

Output:
Group 1
{'group_id': '1', 'severity': 'low', 'status': 'not_a_finding'}
{'group_id': '1', 'severity': 'med', 'status': 'not_a_finding'}
{'group_id': '1', 'severity': 'high', 'status': 'not_a_finding'}

You can create a pandas DataFrame.

df = pd.DataFrame(data)
print(df)

Output:
  group_id severity         status
0        1      low  not_a_finding
1        2      low  not_a_finding
2        3      low  not_a_finding
3        1      med  not_a_finding
4        2      med  not_a_finding
5        3      med  not_a_finding
6        1     high  not_a_finding
7        2     high  not_a_finding
8        3     high  not_a_finding 

[sgtmcc]

Is the format of your text file accurately represented or did you leave some things out?

Yes, it was heavily edited for a much easier read. If The actual text would make things better I can post it. I just put in the relevant parts, such as how I would search and what I would need to replace

The actual file contains 169 entries that I would need to search for, perform a command to verify the setting, and then post the status depending on what the command came back as. Not sure if you or anyone would be familiar but this is for a DOD Stig. I have a working script for the old way of doing it using etree.lxml, but the new way has the entire text in one single line.

Here is a sample of the code:

{"title":"ubuntu_blank","id":"11f4d8bb-0d25-4f7e-b450-eb2260d0e512","stigs":[{"stig_name":"Canonical Ubuntu 20.04 LTS Security Technical Implementation Guide","display_name":"Canonical Ubuntu 20.04 LTS","stig_id":"Canonical_Ubuntu_20-04_LTS_STIG","release_info":"Release: 9 Benchmark Date: 26 Jul 2023","uuid":"5164d686-900f-43a8-8142-5975ee1d576f","reference_identifier":"5318","size":169,"rules":[{"group_id_src":"V-238196","group_tree":[{"id":"V-238196","title":"SRG-OS-000002-GPOS-00002","description":"<GroupDescription></GroupDescription>"}],"group_id":"V-238196","severity":"medium","group_title":"The Ubuntu operating system must provision temporary user accounts with an expiration time of 72 hours or less.","rule_id_src":"SV-238196r653763_rule","rule_id":"SV-238196r653763","rule_version":"UBTU-20-010000","rule_title":"The Ubuntu operating system must provision temporary user accounts with an expiration time of 72 hours or less.","fix_text":"If a temporary account must be created, configure the system to terminate the account after a 72-hour time period with the following command to set an expiration date on it. \n \nSubstitute \"system_account_name\" with the account to be created. \n \n$ sudo chage -E $(date -d \"+3 days\" +%F) system_account_name","weight":"10.0","check_content":"Verify that the Ubuntu operating system expires temporary user accounts within 72 hours or less. \n \nFor every existing temporary account, run the following command to obtain its account expiration information: \n \n$ sudo chage -l system_account_name | grep expires \n \nPassword expires : Aug 07, 2019 \nAccount expires : Aug 07, 2019 \n \nVerify that each of these accounts has an expiration date set within 72 hours of account creation. \n \nIf any temporary account does not expire within 72 hours of that account's creation, this is a finding.","check_content_ref":{"href":"Canonical_Ubuntu_20.04_LTS_STIG.xml","name":"M"},"classification":"Unclassified","discussion":"If temporary user accounts remain active when no longer needed or for an excessive period, these accounts may be used to gain unauthorized access. To mitigate this risk, automated termination of all temporary accounts must be set upon account creation. \n \nTemporary accounts are established as part of normal account activation procedures when there is a need for short-term accounts without the demand for immediacy in account activation. \n \nIf temporary accounts are used, the operating system must be configured to automatically terminate these types of accounts after a DoD-defined time period of 72 hours. \n \nTo address access requirements, many operating systems may be integrated with enterprise-level authentication/access mechanisms that meet or exceed access control policy requirements.","false_positives":"","false_negatives":"","documentable":"false","security_override_guidance":"","potential_impacts":"","third_party_tools":"","ia_controls":"","responsibility":"","mitigations":"","mitigation_control":"","legacy_ids":[],"ccis":["CCI-000016"],"reference_identifier":"5318","uuid":"9090d6fa-64c8-4468-a3ea-31df63c80d07","stig_uuid":"5164d686-900f-43a8-8142-5975ee1d576f","status":"not_reviewed","overrides":{},"comments":"","finding_details":""}

That is just 1 of 169
I would be searching for what I needed to find by the "rule_version":"UBTU-20-010045", and then run a command to check if the computer is compliant, and then annotate the status, and fill in comments. This may be more of a hassle than it's worth. I do appreciate everyone's input

[deanhystad]

That is a json format file. You should load the json file and use the returned object.

with open(filename, "r") as file:
    data = json.load(file)

data will be a python dictionary, and you can navigate through the dictionary using keys.