dict table

[kucingkembar]

sorry for my bad English,
i have this problem, I have this data from JA3

Error:
Id :  AA12
Group :  Firearm - Shotgun
object_class :  Shotgun
RepairCost :  50

Id :  AK47
Group :  Firearm - Assault
Comment :  tier 1 heavy
Reliability :  95

note1 : because for example, I just show a tiny bit of it, real data is above 1 MB of txt
note2 : the data is not properly structured, a grenade launcher data may have an Explosive Area Of Effect, and other weapon data is not

in the end, i like to create a table of it like this

Error:
id    | Group                    | Comment    | RepairCost
AK47 | Firearm - Assault  | tier 1 heavy |
AA12 | Firearm - Shotgun |                     | 50

i stuck with this problem for hours,
can someone give me a clue to do this please,
or at least Google search keyword?

[Pedroski55]

Don't worry about your English, it is good enough!

What is JA3?

Pandas can read a dictionary. What is the problem?

import pandas as pd

data = {'id': [1, 2, 3], 
        'name': ['Alice', 'Bob', 'Charlie'],
        'info': [{'age': 25, 'gender': 'female'},
                 {'age': 30, 'gender': 'male', 'location': 'New York'},
                 {'age': 35, 'gender': 'male', 'location': 'San Francisco'}]}
df = pd.DataFrame(data)
print(df)

Look here.

[kucingkembar]

Don't worry about your English, it is good enough!

What is JA3?

Pandas can read a dictionary. What is the problem?

import pandas as pd

data = {'id': [1, 2, 3], 
        'name': ['Alice', 'Bob', 'Charlie'],
        'info': [{'age': 25, 'gender': 'female'},
                 {'age': 30, 'gender': 'male', 'location': 'New York'},
                 {'age': 35, 'gender': 'male', 'location': 'San Francisco'}]}
df = pd.DataFrame(data)
print(df)

Look here.

thank you for your reply, turn out it need an external package like Panda
can I add data later, but only a part of it, for example
-name Alice, 'location': 'Florida'
-name Alice, 'address': 'Abraham street'
i need the syntax

[Pedroski55]

Of course you can add anything you want!

You do not need pandas to display your data:

data = {'id': [1, 2, 3], 
        'name': ['Alice', 'Bob', 'Charlie'],
        'info': [{'age': 25, 'gender': 'female', 'location': 'Florida', 'Favourite food': 'kangaroo'},
                 {'age': 30, 'gender': 'male', 'location': 'New York', 'language': 'Spanish'},
                 {'age': 35, 'gender': 'male', 'location': 'San Francisco', 'children': 10}]}

from tabulate import tabulate
print(tabulate(data))

[deanhystad]

Avoid using acronyms. JA3: Jagged Alliance 3 or a method for creating SSL/TLS client fingerprints that should be easy to produce on any platform and can be easily shared for threat intelligence. The internet thinks it is the second one.

The file format is odd. Why does a Shotgun have an object class, but not a comment, and an AK47 has a comment, but not an object class. To be of any use, each entry has to have the same attributes. If the file doesn't contain values for all the attributes, it is up to the reader to provide the missing values.

To read the file you might need to look for all the unique tags and create your own reader function. This is a quick pass at a reader for your file.

from io import StringIO
import pandas as pd

data = StringIO("""
Id :  AA12
Group :  Firearm - Shotgun
object_class :  Shotgun
RepairCost :  50

Id :  AK47
Group :  Firearm - Assault
Comment :  tier 1 heavy
Reliability :  95

Id :  Grenade Launcher
Group :  Grenade Launcher
Comment :  Explosive
ExplosiveAreaOfEffect :  10
""")


class Weapon:
    fields = {
        "Id": str,
        "Group": str,
        "Comment": str,
        "object_class": str,
        "RepairCost": int,
        "Reliability": int,
        "ExplosiveAreaOfEffect": int
    }

    def __init__(self, Id: str):
        self.Id = Id

    def __getattr__(self, name):
        """Returns None for missing attributes."""
        if name not in self.fields:
            raise(AttributeError(f"{name} not in Weapon."))
        return self.__dict__.get(name)

    def __setattr__(self, name, value):
        """Only allow setting fields."""
        if self.fields[name] is int:
            value = int(value)
        super().__setattr__(name, value)

    def __str__(self):
        fields = (f"{key}: {value}" for key, value in self.values.items() if value is not None)
        return f"<Weapon {', '.join(fields)}>"

    @property
    def values(self):
        """Returns fields as a dictionary"""
        return {field: getattr(self, field) for field in self.fields}

    @classmethod
    def reader(cls, file):
        """Read weapon information from a file."""
        weapon = None
        for line in file:
            line = line.strip()
            if len(line) == 0:
                continue
            if line.startswith("Id"):
                if weapon:
                    yield weapon
                key, value = map(str.strip, line.split(":"))
                weapon = Weapon(Id=value)
            elif weapon:
                key, value = map(str.strip, line.split(":"))
                weapon.__setattr__(key, value)
        if weapon:
            yield weapon


weapons = list(Weapon.reader(data))
print("Weapons", *weapons, sep="\n", end="\n\n")

df = pd.DataFrame([weapon.values for weapon in weapons])
print(df)
print(df.dtypes)

Output:
Weapons
<Weapon Id: AA12, Group: Firearm - Shotgun, object_class: Shotgun, RepairCost: 50>
<Weapon Id: AK47, Group: Firearm - Assault, Comment: tier 1 heavy, Reliability: 95>
<Weapon Id: Grenade Launcher, Group: Grenade Launcher, Comment: Explosive, ExplosiveAreaOfEffect: 10>

                 Id              Group       Comment object_class  RepairCost  Reliability  ExplosiveAreaOfEffect
0              AA12  Firearm - Shotgun          None      Shotgun        50.0          NaN                    NaN
1              AK47  Firearm - Assault  tier 1 heavy         None         NaN         95.0                    NaN
2  Grenade Launcher   Grenade Launcher     Explosive         None         NaN          NaN                   10.0
Id                        object
Group                     object
Comment                   object
object_class              object
RepairCost               float64
Reliability              float64
ExplosiveAreaOfEffect    float64
dtype: object

The main trick is overriding __getattr__ to return None when the attribute is not found in the object and __setattr__ to only allow setting attributes that are in the fields dictionary. The fields dictionary implements a kind of a typed slots interface.