Python Forum
i want to make a cli command that ...
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
i want to make a cli command that ...
#1
i want to make a cli command that does a combination of what head does and what tail does in posix/unix/linux.  with one stdin input stream it will output some of the beginning and some of the end.  it does onot need to implement the -f option or any others which are better done with the real head and tail commands.  this is to get both the beginning and end of a command's output when piped to it.  if one number is given in the options, it should be the number of lines for both the head and the tail.  if two numbers are given in the options, the first is the number of lines for head and the second is the number of lines for tail.  if the content of a short input stream would be in both the head and tail outputs, then the whole input stream is output, with each line just once.

it might take me a while to get to this project, so if someone else wants to do it, instead ... go for it.

Output:
some command | headandtail -4
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
If you want to do it, then it doesn't matter whether or not I do it, right?  Because you'd still end up doing it yourself anyway :p
Reply
#3
import sys


def get_buffer_size():
    buffer_size = 1
    if len(sys.argv) > 1:
        buffer_size = abs(int(sys.argv[1]))
    return buffer_size


def consume_input(buffer_size):
    buffer = []
    head_sent = False
    for line in sys.stdin:
        buffer.append(line)
        if len(buffer) > buffer_size:
            if not head_sent:
                head_sent = True
                for line in buffer:
                    print(line, end="")
                buffer = []
            else:
                buffer.pop(0)

    if head_sent and buffer:
        print("-" * 78)
    for line in buffer:
        print(line, end="")


if __name__ == "__main__":
    try:
        buffer_size = get_buffer_size()
        consume_input(buffer_size)
    except ValueError as err:
        print("Invalid buffer size.  Any argument to this script should be a number.")
Output:
E:\Projects\etc>pip help | headtail.py Usage: ------------------------------------------------------------------------------ download. Implied with --no-index. E:\Projects\etc>pip help | headtail.py -4 Usage: pip <command> [options] Commands: ------------------------------------------------------------------------------ --disable-pip-version-check Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied with --no-index.
Reply
#4
(Sep-28-2017, 05:39 PM)nilamo Wrote: If you want to do it, then it doesn't matter whether or not I do it, right?  Because you'd still end up doing it yourself anyway :p
perhaps.

(Sep-28-2017, 06:02 PM)nilamo Wrote:
import sys


def get_buffer_size():
buffer_size = 1
if len(sys.argv) > 1:
buffer_size = abs(int(sys.argv[1]))
return buffer_size


def consume_input(buffer_size):
buffer =
head_sent = False
for line in sys.stdin:
buffer.append(line)
if len(buffer) > buffer_size:
if not head_sent:
head_sent = True
for line in buffer:
print(line, end="")
buffer =
else:
buffer.pop(0)

if head_sent and buffer:
print("-" * 78)
for line in buffer:
print(line, end="")


if __name__ == "__main__":
try:
buffer_size = get_buffer_size()
consume_input(buffer_size)
except ValueError as err:
print("Invalid buffer size. Any argument to this script should be a number.")
Output:
E:\Projects\etc>pip help | headtail.py Usage: ------------------------------------------------------------------------------ download. Implied with --no-index. E:\Projects\etc>pip help | headtail.py -4 Usage: pip <command> [options] Commands: ------------------------------------------------------------------------------ --disable-pip-version-check Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied with --no-index.
off by one: i'm getting 1 more line than specified for the head

it looks like quoting source code loses indents.

perhaps i would code it anyway.  coding is fun.  but for headtail, i just want to have the tool.

i guess i will code one of my own to have something in a free license i can give away in a "collection of POSIX commands done in Python" unless nilamo puts his own in such a license (BSD, MIT, GPL, etc.).
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#5
(Sep-29-2017, 05:16 AM)Skaperen Wrote: off by one: i'm getting 1 more line than specified for the head
Line 16 should be >=.  That's my bad for not really testing that hard.

And, unless otherwise stated, feel free to assume all code I post in the forums is released into the public domain.
Reply
#6
Something like this:
#!/usr/bin/env python3


import argparse
import sys


parser = argparse.ArgumentParser()

parser.add_argument('head', type=int, default=4, required=False)
parser.add_argument('tail', type=int, required=False)

args = parser.parse_args()

head = args.head
tail = args.tail

input_ = sys.stdin.read().split('\n')

yellow, purple, default = ('\033[33m', '\033[35m', '\033[0m')

try:
    for line in range(head):
        print('{}{}{}'.format(yellow, input_.pop(0), default))

    if tail:
        for line in range(0 - tail, 0):
            print('{}{}{}'.format(purple, line, default))

except IndexError:
    for line in input_:
        rint('{}{}{}'.format(purple, line, default))
Didn't test it.
Now I see that first as well as second loop could cause the IndexError exception.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply
#7
Quick, how does the real tail handle an input stream larger than system memory?  yes | tail

I don't know what it would do, but reading all of stdin seems like a bad idea.
Reply
#8
It is.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply
#9
(Sep-29-2017, 03:03 PM)nilamo Wrote:
(Sep-29-2017, 05:16 AM)Skaperen Wrote: off by one: i'm getting 1 more line than specified for the head
Line 16 should be >=.  That's my bad for not really testing that hard.

And, unless otherwise stated, feel free to assume all code I post in the forums is released into the public domain. 
now with line 16 changed when i specify 4 i get 3 for head and 3 for tail.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#10
(Sep-29-2017, 08:34 PM)nilamo Wrote: Quick, how does the real tail handle an input stream larger than system memory?  yes | tail

I don't know what it would do, but reading all of stdin seems like a bad idea.
just to be safe, instead of running this on my laptop (not even in a VM on it) i fired up an AWS cloud instance (15GB) and ran i there.  it is still running but i am not sure what it is doing.  both yes and tail have used up about 4GB each (seems a lot for yes) but are not growing  it's not using any swap space, either.  my guess is that the real tail is counting duplicate lines. if yes needs that much space, it must be taken up virtually by the library ... in both processes.  there seems to be no slowdown.

Output:
lt1/forums /home/forums 2> ssh ec2-user@2600:1f18:6291:9700:19e2:fac9:a9fb:19d4 Warning: Permanently added '2600:1f18:6291:9700:19e2:fac9:a9fb:19d4' (ECDSA) to the list of known hosts.        __|  __|_  )        _|  (     /   Amazon Linux AMI       ___|\___|___| https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/ [ec2-user@ip-172-31-0-76 ~]$ yes | tail
and in another window and ssh session (not running slow), two commands separated by some time:
Output:
lt1/forums /home/forums 1> ssh ec2-user@2600:1f18:6291:9700:19e2:fac9:a9fb:19d4 Warning: Permanently added '2600:1f18:6291:9700:19e2:fac9:a9fb:19d4' (ECDSA) to the list of known hosts. Last login: Sun Oct  1 05:43:55 2017 from 2a02:348:61:5d75::f        __|  __|_  )        _|  (     /   Amazon Linux AMI       ___|\___|___| https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/ [ec2-user@ip-172-31-0-76 ~]$ ps awux|egrep 'yes|tail'|fgrep -v 'grep' ec2-user  3171 99.9  0.0   4316   708 pts/0    S+   05:21  24:48 yes ec2-user  3172 51.2  0.0   4344   732 pts/0    R+   05:21  12:42 tail [ec2-user@ip-172-31-0-76 ~]$ ps awux|egrep 'yes|tail'|fgrep -v 'grep' ec2-user  3171 99.9  0.0   4316   708 pts/0    R+   05:21  35:21 yes ec2-user  3172 51.1  0.0   4344   732 pts/0    S+   05:21  18:06 tail [ec2-user@ip-172-31-0-76 ~]$

i killed yes and tail output 10 lines of "y" (as expected).
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  i need to make a command that ... Skaperen 5 2,781 Nov-30-2019, 09:20 AM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020