Python Forum

Hello,

I found the script online that uploads multiple JPG files. I changed it so I can upload multiple BLOB files, i.e.) csv, xls, etc. However, I am receiving the following error message when I run the script:

Error:
azure.core.exceptions.HttpResponseError: The specifed resource name contains invalid characters.
RequestId:62
Time:2023-12-21T15:42:33.3507947Z
ErrorCode:InvalidResourceName
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:62
Time:2023-12-21T15:42:33.3507947Z</Message></Error>

The file names are lowercase and alphanumeric: bcp-notes-batch-0.csv

Here is the script:

# Replace with blob container
MY_IMAGE_CONTAINER = "source_system"

# Replace with the local folder which contains the files for upload
LOCAL_FILE_PATH = 'I:/bcp'

class AzureBlobFileUploader:
    def __init__(self):
        print("Initializing AzureBlobFileUploader")
        # Initialize the connection to Azure storage account
        self.blob_service_client = BlobServiceClient.from_connection_string(MY_CONNECTION_STRING)

    def upload_all_files_in_folder(self):
        all_file_names = [f for f in os.listdir(LOCAL_FILE_PATH)
                          if os.path.isfile(os.path.join(LOCAL_FILE_PATH, f))]
        result = self.run(all_file_names)
        print(result)

    def run(self, all_file_names):
        with ThreadPool(processes=int(10)) as pool:
            return pool.map(self.upload_file, all_file_names)

    def upload_file(self, file_name):
        blob_client = self.blob_service_client.get_blob_client(container=MY_IMAGE_CONTAINER, blob=file_name)
        upload_file_path = os.path.join(LOCAL_FILE_PATH, file_name)
        content_settings = ContentSettings(content_type='application/octet-stream')
        print(f"Uploading file - {file_name}")
        with open(upload_file_path, "rb") as data:
            blob_client.upload_blob(data, overwrite=True, content_settings=content_settings)
        return file_name

# Initialize class and upload files
azure_blob_file_uploader = AzureBlobFileUploader()
azure_blob_file_uploader.upload_all_files_in_folder()

I would start with replacing the list comprehension on line 14 with a regular loop. Catch the exception, and when it happens, print out the filename with repr() in case it has weird things that aren't obvious.

These are the only two files in the directory. All characters in the name are valid:

Error:
Initializing AzureBlobFileUploader
Uploading file - ibrtfnotesbatch00.csv
Uploading file - ibrthnotesbatch01.csv
Exception uploading file - 'ibrthnotesbatch01.csv': The specifed resource name contains invalid characters.
RequestId:70
Time:2023-12-21T18:18:25.7180043Z
ErrorCode:InvalidResourceName
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:70
Time:2023-12-21T18:18:25.7180043Z</Message></Error>
Exception uploading file - 'ibrtfnotesbatch00.csv': The specifed resource name contains invalid characters.
RequestId:28
Time:2023-12-21T18:18:25.7217421Z
ErrorCode:InvalidResourceName
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:28
Time:2023-12-21T18:18:25.7217421Z</Message></Error>
[None, None]

According to azure documentation:

Quote:A path name is composed of one or more path name components (directory or file name) separated by the forward-slash (/) character. All path name components other than the last path name component denote directories. The last path name component denotes a directory or a file.

Is the problem that os.path.join(LOCAL_FILE_PATH, file_name) uses '\'?

(Dec-21-2023, 06:50 PM)deanhystad Wrote: [ -> ]According to azure documentation:

Quote:A path name is composed of one or more path name components (directory or file name) separated by the forward-slash (/) character. All path name components other than the last path name component denote directories. The last path name component denotes a directory or a file.

Is the problem that os.path.join(LOCAL_FILE_PATH, file_name) uses '\'?

Still no luck:

class AzureBlobFileUploader:
    def __init__(self):
        print("Initializing AzureBlobFileUploader")
        self.blob_service_client = BlobServiceClient.from_connection_string(MY_CONNECTION_STRING)

    def upload_all_files_in_folder(self):
        all_file_names = [f for f in os.listdir(LOCAL_FILE_PATH) if os.path.isfile(os.path.join(LOCAL_FILE_PATH, f))]
        result = self.run(all_file_names)
        print(result)

    def run(self, all_file_names):
        with ThreadPool(processes=int(10)) as pool:
            return pool.map(self.upload_file, all_file_names)

    def upload_file(self, file_name):
            try:
                print(repr(file_name))  # Print the repr() of the file name
                blob_client = self.blob_service_client.get_blob_client(container=MY_IMAGE_CONTAINER, blob=file_name)
                upload_file_path = os.path.join(LOCAL_FILE_PATH, file_name).replace('\\', '/')  # Replace backslashes
                print(f"Uploading file - {file_name}")
                with open(upload_file_path, "rb") as data:
                    blob_client.upload_blob(data, overwrite=True)
                return file_name
            except Exception as e:
                print(f"Exception uploading file - {repr(file_name)}: {e}")

    def get_containers(self):
        try:
            containers = self.blob_service_client.list_containers()
            print([container.name for container in containers])
        except Exception as e:
            print(e.message)

# Initialize class and upload files
azure_blob_file_uploader = AzureBlobFileUploader()
azure_blob_file_uploader.get_containers()
azure_blob_file_uploader.upload_all_files_in_folder()

Error:
Initializing AzureBlobFileUploader
'ibrtfnotesbatch00.csv'
'ibrthnotesbatch01.csv'
Uploading file - ibrtfnotesbatch00.csv
Uploading file - ibrthnotesbatch01.csv
Exception uploading file - 'ibrtfnotesbatch00.csv': The specifed resource name contains invalid characters.
RequestId:1b87ffae-901e-0077-4242-34969d000000
Time:2023-12-21T19:17:32.3704303Z
ErrorCode:InvalidResourceName
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:1b87ffae-901e-0077-4242-34969d000000
Time:2023-12-21T19:17:32.3704303Z</Message></Error>
Exception uploading file - 'ibrthnotesbatch01.csv': The specifed resource name contains invalid characters.
RequestId:74f78216-701e-0040-3a42-344431000000
Time:2023-12-21T19:17:41.6794092Z
ErrorCode:InvalidResourceName
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:74f78216-701e-0040-3a42-344431000000
Time:2023-12-21T19:17:41.6794092Z</Message></Error>
[None, None]

I looked at the problem wrong. The \ has nothing to do with the issue because that is a local filename, not the name of the blob resource. I think our problem is the ".". From the azure docs

Quote:Azure Storage resource names must follow certain rules, such as:

Must be between 3 and 63 characters long.
Must start with a letter or number.
Can only contain lowercase letters, numbers, and hyphens (-).
Must not start or end with a hyphen (-).

I don't know what azure storage is. If you need to change file names to suit, that can be done.

I believe rsync will work under MS Windoze, but I have never tried.

For syncing files I don't think rsync can be beat. Just use it on the command line, or make a bash script to copy from multiple folders.

You must have write permission on the remote machine.

For example, to send all files from /home/pedro/jpgs to another machine, you will be asked for your user, in this case pedro's, password.

If the remote machine doesn't have the directory /home/pedro/jpgs, it will be created, as will any and all sub directories.

rsync will only send (or receive) files which have changed since the last time you used rsync. This one-liner sends files over my LAN to another laptop, but I could put the address of my little cloud server. I just need to know the ip4 address of the other machine:

Quote:rsync -av -e "ssh" --progress /home/pedro/jpgs [email protected]:/home/pedro/

You can even use rsync to send files to a usb stick!

I believe -e "ssh" is redundant now, as all transfers use ssh, but I leave it there anyway!