|
Search Results
|
| Post |
Author |
Forum |
Replies |
Views |
Posted
[asc]
|
| |
|
Thread: request.get to read large response
Post: RE: request.get to read large response
| It is working with just request.get without json_stream. it was timing issue.
I have added timeout=200 and I got the whole result back.
Thank you everyone who help. |
|
pythonlearner1 |
General Coding Help |
7 |
3,213 |
Apr-05-2022, 08:21 PM |
| |
|
Thread: request.get to read large response
Post: RE: request.get to read large response
| I try two method describe in that document but didn't work that is why ask. searching around didn't find any example either.
here is what I try
# Option 1: supply json_stream.encoding.default as th... |
|
pythonlearner1 |
General Coding Help |
7 |
3,213 |
Apr-05-2022, 06:26 PM |
| |
|
Thread: request.get to read large response
Post: RE: request.get to read large response
| (Apr-05-2022, 02:24 PM)buran Wrote: Did you look at Streaming requests? Would it work in your case?
Or a package like json_stream?
Great, json_stream worked. now how do I encode it
with re... |
|
pythonlearner1 |
General Coding Help |
7 |
3,213 |
Apr-05-2022, 04:56 PM |
| |
|
Thread: request.get to read large response
Post: RE: request.get to read large response
| (Apr-05-2022, 08:11 AM)ibreeden Wrote: But what is the output of your code? Does it show the error message or the json?
if I pull one record it shows in json format.
if I do try to pull all the rec... |
|
pythonlearner1 |
General Coding Help |
7 |
3,213 |
Apr-05-2022, 02:08 PM |
| |
|
Thread: request.get to read large response
Post: request.get to read large response
| I am trying to use curl to read read large response.
I ran that from command line and redirected to text file and it is 30Mb file
using python I can put one record at a time but not everything that ... |
|
pythonlearner1 |
General Coding Help |
7 |
3,213 |
Apr-05-2022, 12:26 AM |
| |
|
Thread: Convert to UTC without changing time
Post: RE: Convert to UTC without changing time
| used this to get what I wated
mytime_timestamp = datetime.strptime(mytime, "%Y-%m-%d")
datetime_obj_utc = mytime_timestamp.replace(tzinfo=timezone('Etc/UTC'))
datetime_obj_cst = mytime_timestamp.repl... |
|
pythonlearner1 |
General Coding Help |
1 |
2,120 |
Nov-05-2020, 09:54 PM |
| |
|
Thread: Convert to UTC without changing time
Post: Convert to UTC without changing time
| I am pulling data from elasticsearch in d/m/yyyy format.
when it come I believe it is in UTC format. after some processing I am putting that data back in to Elasticsearch and it converts it to UTC an... |
|
pythonlearner1 |
General Coding Help |
1 |
2,120 |
Nov-05-2020, 02:42 PM |
| |
|
Thread: bulk update in elasticsearch
Post: RE: bulk update in elasticsearch
| I created list something like this by reading data from mongo.
where I am intested in updating record with new value called site
{'_op_type': 'update', '_index': 'my_index-2020', '_id': 314800814, ... |
|
pythonlearner1 |
General Coding Help |
1 |
6,002 |
Jun-10-2020, 10:01 PM |
| |
|
Thread: bulk update in elasticsearch
Post: bulk update in elasticsearch
| I am trying to update bulk document to elasticsearch
I am able to update this one by one document
following code works
elastic_output = Elasticsearch(output_master, http_auth=(elastic_user, el... |
|
pythonlearner1 |
General Coding Help |
1 |
6,002 |
Jun-10-2020, 08:05 PM |
| |
|
Thread: Need help really bad ;(
Post: RE: Need help really bad ;(
| (May-26-2020, 12:12 PM)buran Wrote: time.sleep(5)
But I never understood why every other newbie wants to delay the program execution with unnecessary delays like this
because he/she might want to ch... |
|
pythonlearner1 |
General Coding Help |
7 |
3,227 |
May-26-2020, 08:39 PM |
| |
|
Thread: Parse JSON multiple objects
Post: RE: Parse JSON multiple objects
| print(type(data))
will tell you which type of variable is data
string, integer, json etc.. |
|
pythonlearner1 |
General Coding Help |
8 |
5,754 |
May-26-2020, 08:37 PM |
| |
|
Thread: Add variable value to string
Post: RE: Add variable value to string
| Thank you Buran, That is what I needed. yes I never thought that I am setting up json. still stuck in old way of doing (bash scripting) |
|
pythonlearner1 |
General Coding Help |
5 |
2,989 |
May-25-2020, 04:44 PM |
| |
|
Thread: Add variable value to string
Post: RE: Add variable value to string
| I did do this and it worked. but now thinking if I can do this in one line
data1 = '\n{ "objects":[\n {\n "type":"dashboard",\n "id":'
data2 = '\n } ] ,\n "includeReferencesDeep": true\... |
|
pythonlearner1 |
General Coding Help |
5 |
2,989 |
May-24-2020, 09:12 PM |
| |
|
Thread: Add variable value to string
Post: Add variable value to string
| I am creating variable
dashboard_id="66862fa0-dafa-4d28572533a"I want to create a another variable and insert this value to it
Output:{ "objects":[
{
"type":"dashboard",
"id":"66862fa0-dafa-4d... |
|
pythonlearner1 |
General Coding Help |
5 |
2,989 |
May-24-2020, 08:47 PM |
| |
|
Thread: json to ndjson
Post: RE: json to ndjson
| Thank you both of you
This is how my code is
response = requests.get(url, params=params, auth=('user', 'pass'))#
print (response.headers['content-type']) --> this says 'application/json; charset=u... |
|
pythonlearner1 |
General Coding Help |
5 |
9,636 |
May-22-2020, 02:29 PM |
| |
|
Thread: json to ndjson
Post: json to ndjson
| I have a output from request.get it is something like this
actual output is long and multiple of this line.
[{"a":1,"b":2,"c":3},{"x":4,"y":5,"z":6}]
i want this to convert to something like this
... |
|
pythonlearner1 |
General Coding Help |
5 |
9,636 |
May-21-2020, 10:32 PM |
| |
|
Thread: date time question
Post: RE: date time question
| this is good. it is working
but I want to keep date and time 20:10:10 as is and then tell it that this is UTC timezone.
I just have play around with different setting.
as I want date-time as is and... |
|
pythonlearner1 |
General Coding Help |
2 |
2,325 |
Jan-31-2020, 08:01 PM |
| |
|
Thread: date time question
Post: date time question
| I would like to convert string in to date format which is acceptable by elasticsearch
doc = {
'submitted': '2020-01-25 20:10:10',
'num': '2',
'timestamp': 1579624522874,
}
I have two fie... |
|
pythonlearner1 |
General Coding Help |
2 |
2,325 |
Jan-31-2020, 06:11 PM |
