Python Forum
Problem with bash shell needed for my python project
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Problem with bash shell needed for my python project
#1
Forgive me, There is no python in the following, though there will be (part of my next project).

Since I am so familiar with this forum, I thought I'd ask here as one of you may have already solved this issue.
As input to my python project, I need csv files that contain town and city names from each country / state, with osm node id's, and
latitude and longitude.
I will be most likely using osmnx package eventually, but for now a bash shell script will do.
aparently it's using the filter as the output file.

The problem is, I can only get the bash shell to partially work, and can't see why the final half does not work (the osmconvert part)
here's the script:
#!/bin/bash

cd Alabama
# osmosis binary
OSMOSIS="/usr/bin/osmosis"
# accepts PBF files
INPUT="Alabama-Latest.Osm.Pbf"
# outputs to OSM XML file
OUTPUT="AlabamaCities.pbf"
# tag to use for filter
FILTER="place=city,town,village,hamlet,suburb,borough,municipality"

# osmconvert binary
OSMCONVERT="/usr/bin/osmconvert"
# accepts PBF files
OSMCINPUT="AlabamaCities.pbf"
# output csv file
CSVOUT="AlabamaCities.csv"
# tag to use for filter
FILTER1="""@id @lat @lon name value"""

$OSMOSIS \
  --read-pbf $INPUT \
  --tf reject-relations \
  --tf reject-ways \
  --tf accept-nodes $FILTER \
  --write-xml $OUTPUT \
  | osmconvert - \
  --all-to-nodes \
  --csv=$FILTER1 \
  -o=$CSVOUT

cd ..
I tried running without actually changing my directory, but read 'somewhere' that osmconvert wants to be run from where the files reside, thus the 'cd' commands
The extract from the main .pbf file works flawlessly, there is no indication that the osmconvert does not work, but in the end, there is no .csv file.

here's the output of the bash shell:
Output:
osmconvert Error: could not open input file: @lat Jun 21, 2021 2:56:14 PM org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.47 Jun 21, 2021 2:56:15 PM org.openstreetmap.osmosis.core.Osmosis run INFO: Preparing pipeline. Jun 21, 2021 2:56:15 PM org.openstreetmap.osmosis.core.Osmosis run INFO: Launching pipeline execution. Jun 21, 2021 2:56:15 PM org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline executing, waiting for completion. Jun 21, 2021 2:56:19 PM org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline complete. Jun 21, 2021 2:56:19 PM org.openstreetmap.osmosis.core.Osmosis run INFO: Total execution time: 4199 milliseconds.
I can run the following from the command line, and it properly produces a .csv file:
$ osmconvert AlabamaCities.pbf --csv="@id @lat @lon name" -o=AlabamaCities.csv

The NewHampshire.osm.pbf file can be obtained here
(85.7Mb)
it can be converted to osm whit this command: osmconvert alabama-latest.osm.pbf alabama-latest.osm
Reply
#2
Have you tried this perhaps ?
FILTER1="\"@id @lat @lon name value\""
Another idea would be to write a xonsh script where you can mix shell and python.
Reply
#3
That didn't do it.
I was once a whizz at bash and c shell.
I find that since I started using python, I use shell scripts less and less.
Reply
#4
This seems to give better results
#!/bin/bash
FILTER1="@id @lat @lon name value"

/home/eric/bin/echoargs --csv="$FILTER1"
Output:
~/P/S/2021-01 paillasse/pf/script.sh ['/home/eric/bin/echoargs', '--csv=@id @lat @lon name value']
I used here my awesome 'echoargs' program to check command line arguments:
#!/usr/bin/env python3
# echoargs script
import sys

if __name__ == '__main__':
    print(sys.argv)
Reply
#5
I settled for:

#!/bin/bash

OSMOSIS="/usr/bin/osmosis"
OSFILTER="place=city,town,village,hamlet,suburb,borough,municipality"
OCFILTER="@id @lat @lon name"

cd Alabama
$OSMOSIS --read-pbf "Alabama-Latest.Osm.Pbf" --tf reject-relations --tf reject-ways --tf accept-nodes $OSFILTER --write-xml "AlabamaCities.pbf"
osmconvert AlabamaCities.pbf --csv="$OCFILTER" -o="AlabamaCities.csv"
cd ..
which gets the job done for now. I am in the process of converting to 100% python now.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  unix shell in python Skaperen 3 2,552 Mar-29-2019, 12:02 AM
Last Post: snippsat
  BASH ? elico 2 4,405 Oct-27-2016, 12:18 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020