Python Forum

Hi,

Help with creating a control record as a program variable.

I am writing a script to call ~10 APIs and analyse the data on frequent basis (>10 & < 60+ times an hour). The APIs can take 0.5 to 3 seconds each. Each API represents a different timeframe from a Month down to a Minute, so if the timeframe is still current and therefore the data has not changed since the last update (most likely Monthly, Weekly and Daily) then I want to avoid calling the API as well as analysing the data, by using the existing data that was previously extracted and analysed (This will be in memory as the program will still be active).

I planned to do this by keeping a control record with a 'timestamp' for each API. Element 1 being the Month timeframe and the last element being the one minute timeframe. The record being updated by a new timestamp in the corresponding element when an update has occured.

My limited "understanding" of lists, is that inserting a an element into a specific position with cause the new element to force some elements further down the list, thus changing their position.

So the array would look like this:

api_control [ timestamp, timestamp, timestamp etc.]

Where the first timestamp would refer to Months, the second to weeks, then days, 12 hours, 8 hours.... 5 minutes.

Ditto "understanding" - Tuples on the surface, may not be best suited, but see caveat!

Obviously I could keep a number of variables, one for each timeframe, but this looks like it is unnecessarily messy etc. etc.

I would be grateful for thoughts / a pointer to a "workable solution" 

Thanks

Bass

This post is all over the place, I'm not sure what exactly the problem is. Could you rephrase your question more succinctly? Ideally simple descriptions of data before and after an operation.

Hi micseydel,

Thank you for your reply, apologies my brain is fried at the moment.

In summary:

1. Overall Requirement - I am trying to find a way to store the timestamps of the various API data requests, to enable incremental updates and analysis, i.e. to enable the script to choose which API's need to be called and processed. Processing a full update every time would be prohibitive.

2. What I have Tried - I have listed some of the methods that I have considered but that have not been successful. Rather than just list the problem.

3. Specific Requirement - In other words, I am looking for a method of building, reading and updating an array, where the elements remain in the same position, i.e. element one would always be the "Month" data timestamp and element two would always be the "Week" data timestamp etc.

4. The Data Structure - Each time the script is run, the array would start off null, as each API is executed, the timestamp would be inserted into the relevant time frame position The data in the array would be a list of 10 timestamps (Unix/POSIX) Format. As an update is requested by the user, the system would look at the array and update only the timeframes that needed updating, e.g. the "Daily" data would need to be refreshed at midnight, the "Hourly" data once the time has reached the top of the clock. If the "Hourly" data was last accessed at 12:45pm and it is now 12:55pm then no update would be needed, however if it was past 1pm then the routinme would need to update the "Hour" data.

Hope this makes better sense.

Bass

I would look for memoizing decorator

Thanks volcano,

Have googled this and found lots of useful stuff.

Thanks

Bass