Python Forum
Generate Test data (.csv) using Pandas
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Generate Test data (.csv) using Pandas
I want to generate the test data in (.csv format) using Python.
Below is my script using pandas but I'm stuck at randomly generating test data for a column called ACTIVE.
1. ACTIVE column should have value only 0 and 1.
2. Also another issue is that how can I have data of array of varying length.

Thank you in advance.

import pandas as pd
import numpy as np
import random

x = str(input('Enter the date: '))
y = ['1', '0']
data = {'ACCOUNT': ['', 'Enabled', 'Disabled', 'Hold'],
        'CUSTOMER NAME': ['Test Name1', 'Test Name2']}

df = pd.DataFrame(data, columns=['ACCOUNT NUMBER', 'ACCOUNT', 'CUSTOMER NAME', 'ACTIVE', 'DATE'])
df['ACCOUNT NUMBER'] = 123  #(This needs to auto-increment)
df['ACCOUNT NUMBER'] = 123
df['ACTIVE'] = random.choice(y) #(how column named active should randomly take value 0 or 1)
df['DATE'] = x
df.to_csv(r'C:\Users\Test_User\Desktop\TestFolder\TestFile.csv', index=False)
Enter the date: 9/9/2020 Traceback (most recent call last): File "C:/Users/TestUser/PycharmProjects/TestDataAutomation/", line 10, in <module> df = pd.DataFrame(data, columns=['ACCOUNT NUMBER', 'ACCOUNT', 'CUSTOMER NAME', 'ACTIVE', 'DATE']) File "C:\Users\ TestUser\AppData\Roaming\Python\Python37\site-packages\pandas\core\", line 435, in __init__ mgr = init_dict(data, index, columns, dtype=dtype) File "C:\Users\ TestUser\AppData\Roaming\Python\Python37\site-packages\pandas\core\internals\", line 228, in init_dict index = extract_index(arrays[~missing]) File "C:\Users\ TestUser\AppData\Roaming\Python\Python37\site-packages\pandas\core\internals\", line 365, in extract_index raise ValueError("arrays must all be same length") ValueError: arrays must all be same length Process finished with exit code 1
Pandas dataframes don't have "shaggy bottoms" - columns must be of the same length. You can either extend the short columns with null values or use multiple dataframes.
Thanks for replying. I was seeing if there's any other way to do it which i am not aware of.

Another question: How can I auto-increment a column in Pandas using data frame.
If I've column say "ID" and if I set initial value of id = 10 then I want it to auto-increment from 10, 11, 12 etc.

Not quite clear on what you want to do. You want to increment all values in a column? Increment the values in each row of a single column? Move from column to column? Not sure what you mean by autoincrement in regards to a column.
I have a column called "Account Number". I want that column to auto-increment. Let's say I have Account Number value is set to 1000 then I want that value to be auto increment for other rows in same column. Ex below:

Account Number
Couple ways to do it. Purely Pandas would try this
start = 1000
df['autoinc'] = pd.RangeIndex(stop=df.shape[0])+start
It will work without the "start", did not have a dataframe handy to test by adding the start.

Possibly Related Threads…
Thread Author Replies Views Last Post
Thumbs Up can't access data from URL in pandas/jupyter notebook aaanoushka 1 618 Feb-13-2022, 01:19 PM
Last Post: jefsummers
Question Sorting data with pandas TheZaind 4 1,064 Nov-22-2021, 07:33 PM
Last Post: aserian
  Pandas Data frame column condition check based on length of the value aditi06 1 1,360 Jul-28-2021, 11:08 AM
Last Post: jefsummers
  [Pandas] Write data to Excel with dot decimals manonB 1 2,169 May-05-2021, 05:28 PM
Last Post: ibreeden
  pandas.to_datetime: Combine data from 2 columns ju21878436312 1 1,481 Feb-20-2021, 08:25 PM
Last Post: perfringo
  Mann Whitney U-test on several data sets rybina 2 1,201 Jan-05-2021, 03:08 PM
Last Post: rybina
  pandas read_csv can't handle missing data mrdominikku 0 1,422 Jul-09-2020, 12:26 PM
Last Post: mrdominikku
  Pandas data frame creation from Kafka Topic vboppa 0 1,117 Jul-01-2020, 04:23 PM
Last Post: vboppa
  Read json array data by pandas vipinct 0 1,107 Apr-13-2020, 02:24 PM
Last Post: vipinct
  add formatted column to pandas data frame alkaline3 0 1,002 Mar-22-2020, 06:44 PM
Last Post: alkaline3

Forum Jump:

User Panel Messages

Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020