Python Forum
Help copying a column from a csv to another file with some extras
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help copying a column from a csv to another file with some extras
#1
Hello,

I need to copy the contents of a .csv, that gets dumped from SQL each night to a folder location on my Ubuntu server that contains 100s of IP addresses to an area within a .conf file. Well I'm happy to create a new file rather than try and open an existing file and insert.

I'm a novice at Python, but getting this working will help. VScode is amazing to.

The csv puts all the IP addresses on the first column:

ip_address
192.168.0.1
192.168.0.2
192.168.0.3
192.168.0.4 
ect...
And I need it to put into a file called something like routers.conf like this on row 2, but I need it be formatted like below and include the " instead of 'around the IP, add the :161 etc and inject into row 2 and replace what is there. Plus add the rest of the config.

[[inputs.snmp]]
  agents ["192.168.0.1:161","192.168.0.2:161","192.168.0.3:161","192.168.0.4:161"]
  version = 2
  community = "publicblah"
  interval = "120s"
  timeout = "5s"
  retries = 0
Could Python do this?

I'm currently using this to open and view it:

    with open("ip.csv", 'r', encoding='utf-8-sig') as infile:
    reader = csv.reader(infile, delimiter=" ")
    header = next(reader)
    print(header)
I've been reading through - https://realpython.com/python-csv/

I did have a play with the outfile = open and outfile.write options, but get lost as this just writes a new .csv.

Any help to copy this data, format it, and create a new file with the addition config file would be most welcome.
Reply
#2
Is the delimiter a whitespace? If it's the case, you can use split instead.
Here is an example with csv, where the delimiter is a ,.

To validate an IP-Address, you can use ipaddress.ip_address.
If the str is not a valid IPv4- or IPv6-Address, a ValueError is thrown.

IPv4Address/IPv6Address are sortable.


import csv
from collections.abc import Generator
from ipaddress import ip_address
from pathlib import Path


template = """
[[inputs.snmp]]
  agents [{ips}]
  version = 2
  community = "publicblah"
  interval = "120s"
  timeout = "5s"
  retries = 0
"""


def reader(file: str | Path) -> Generator[str, None, None]:
    with open(file, newline="") as fd:
        for ip, *_ in csv.reader(fd):
            # validation of ip
            try:
                ip = ip_address(ip)
            except ValueError:
                print(f"Invalid IP: '{ip}' skipping...")
                continue

            yield ip


def make_conf(infile: str | Path, outfile: str | Path) -> None:
    ips = [f'"{ip}"' for ip in sorted(reader(infile))]
    with open(outfile, "w") as fd_out:
        fd_out.write(template.format(ips=",".join(ips)))


if __name__ == "__main__":
    infile = Path.home().joinpath("Desktop", "data.csv")
    outfile = Path.home().joinpath("Desktop", "snmp.conf")
    make_conf(infile, outfile)
Pedroski55 likes this post
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#3
@DeaD_EyE

In

def reader(file: str | Path) -> Generator[str, None, None]
What does this do?

Quote:Generator[str, None, None]

Do str, None, None represent send(), throw() and close()??

The function reader works without Generator being explicitly declared. Is that just extra error catching?
Reply
#4
It's a function definition with TypeHints for arguments and the return type.
After the colon are the expected types.

It helps the IDE to know the expected types. This code does not do any checks during runtime.

def reader(file: str | Path) -> Generator[str, None, None]
The first argument should be a str or path. The return type is a generator, which yields str. The Generator does not have a SendType nor a ReturnType.

Syntax for Generator
Generator[YieldType, SendType, ReturnType]
The same signature without TypeHints is easier to understand:
def reader(file):
The whole Code without TypeHints is shorter:
import csv
from ipaddress import ip_address
from pathlib import Path
 
 
template = """
[[inputs.snmp]]
  agents [{ips}]
  version = 2
  community = "publicblah"
  interval = "120s"
  timeout = "5s"
  retries = 0
"""
 
 
def reader(file):
    with open(file, newline="") as fd:
        for ip, *_ in csv.reader(fd):
            # validation of ip
            try:
                ip = ip_address(ip)
            except ValueError:
                print(f"Invalid IP: '{ip}' skipping...")
                continue
 
            yield ip
 
 
def make_conf(infile, outfile):
    ips = [f'"{ip}"' for ip in sorted(reader(infile))]
    with open(outfile, "w") as fd_out:
        fd_out.write(template.format(ips=",".join(ips)))
 
 
if __name__ == "__main__":
    infile = Path.home().joinpath("Desktop", "data.csv")
    outfile = Path.home().joinpath("Desktop", "snmp.conf")
    make_conf(infile, outfile)
Pedroski55 and carecavoador like this post
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to read csv file update matplotlib column chart regularly SamLiu 2 1,234 Jan-21-2023, 11:33 PM
Last Post: SamLiu
  Read xml column inside csv file with Python estertabita 2 1,485 Jul-26-2022, 06:09 PM
Last Post: Larz60+
  How to split file by same values from column from imported CSV file? Paqqno 5 3,131 Mar-24-2022, 05:25 PM
Last Post: Paqqno
  Appending Excel column value as CSV file name sh1704 0 1,420 Feb-06-2022, 10:32 PM
Last Post: sh1704
  Create zip file from the BLOB column in ORACLE DB nnsatpute 2 2,248 Dec-31-2021, 11:00 AM
Last Post: ibreeden
  split txt file data on the first column value shantanu97 2 2,634 Dec-29-2021, 05:03 PM
Last Post: DeaD_EyE
  [Solved] Reading every nth line into a column from txt file Laplace12 7 5,591 Jun-29-2021, 09:17 AM
Last Post: Laplace12
  Copy column from one existing excel file to another file mkujawsk 0 6,063 Apr-14-2021, 06:33 PM
Last Post: mkujawsk
  Remove single and double quotes from a csv file in 3 to 4 column shantanu97 0 7,271 Mar-31-2021, 10:52 AM
Last Post: shantanu97
  Add a new column when I extract each sheet in an Excel workbook as a new csv file shantanu97 0 2,352 Mar-24-2021, 04:56 AM
Last Post: shantanu97

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020