i want to make a
cli command that does a combination of what
head does and what
tail does in posix/unix/linux. with one stdin input stream it will output some of the beginning and some of the end. it does onot need to implement the -f option or any others which are better done with the real
head and
tail commands. this is to get both the beginning and end of a command's output when piped to it. if one number is given in the options, it should be the number of lines for both the head and the tail. if two numbers are given in the options, the first is the number of lines for head and the second is the number of lines for tail. if the content of a short input stream would be in both the head and tail outputs, then the whole input stream is output, with each line just once.
it might take me a while to get to this project, so if someone
else wants to do it, instead ... go for it.
Output:
some command | headandtail -4
If you want to do it, then it doesn't matter whether or not I do it, right? Because you'd still end up doing it yourself anyway :p
import sys
def get_buffer_size():
buffer_size = 1
if len(sys.argv) > 1:
buffer_size = abs(int(sys.argv[1]))
return buffer_size
def consume_input(buffer_size):
buffer = []
head_sent = False
for line in sys.stdin:
buffer.append(line)
if len(buffer) > buffer_size:
if not head_sent:
head_sent = True
for line in buffer:
print(line, end="")
buffer = []
else:
buffer.pop(0)
if head_sent and buffer:
print("-" * 78)
for line in buffer:
print(line, end="")
if __name__ == "__main__":
try:
buffer_size = get_buffer_size()
consume_input(buffer_size)
except ValueError as err:
print("Invalid buffer size. Any argument to this script should be a number.")
Output:
E:\Projects\etc>pip help | headtail.py
Usage:
------------------------------------------------------------------------------
download. Implied with --no-index.
E:\Projects\etc>pip help | headtail.py -4
Usage:
pip <command> [options]
Commands:
------------------------------------------------------------------------------
--disable-pip-version-check
Don't periodically check PyPI to determine
whether a new version of pip is available for
download. Implied with --no-index.
(Sep-28-2017, 05:39 PM)nilamo Wrote: [ -> ]If you want to do it, then it doesn't matter whether or not I do it, right? Because you'd still end up doing it yourself anyway :p
perhaps.
(Sep-28-2017, 06:02 PM)nilamo Wrote: [ -> ]import sys
def get_buffer_size():
buffer_size = 1
if len(sys.argv) > 1:
buffer_size = abs(int(sys.argv[1]))
return buffer_size
def consume_input(buffer_size):
buffer =
head_sent = False
for line in sys.stdin:
buffer.append(line)
if len(buffer) > buffer_size:
if not head_sent:
head_sent = True
for line in buffer:
print(line, end="")
buffer =
else:
buffer.pop(0)
if head_sent and buffer:
print("-" * 78)
for line in buffer:
print(line, end="")
if __name__ == "__main__":
try:
buffer_size = get_buffer_size()
consume_input(buffer_size)
except ValueError as err:
print("Invalid buffer size. Any argument to this script should be a number.")
Output:
E:\Projects\etc>pip help | headtail.py
Usage:
------------------------------------------------------------------------------
download. Implied with --no-index.
E:\Projects\etc>pip help | headtail.py -4
Usage:
pip <command> [options]
Commands:
------------------------------------------------------------------------------
--disable-pip-version-check
Don't periodically check PyPI to determine
whether a new version of pip is available for
download. Implied with --no-index.
off by one: i'm getting 1 more line than specified for the head
it looks like quoting source code loses indents.
perhaps i would code it anyway. coding is fun. but for headtail, i just want to have the tool.
i guess i will code one of my own to have something in a free license i can give away in a "collection of POSIX commands done in Python" unless nilamo puts his own in such a license (BSD, MIT, GPL, etc.).
(Sep-29-2017, 05:16 AM)Skaperen Wrote: [ -> ]off by one: i'm getting 1 more line than specified for the head
Line 16 should be
>=
. That's my bad for not really testing that hard.
And, unless otherwise stated, feel free to assume all code I post in the forums is released into the public domain.
Something like this:
#!/usr/bin/env python3
import argparse
import sys
parser = argparse.ArgumentParser()
parser.add_argument('head', type=int, default=4, required=False)
parser.add_argument('tail', type=int, required=False)
args = parser.parse_args()
head = args.head
tail = args.tail
input_ = sys.stdin.read().split('\n')
yellow, purple, default = ('\033[33m', '\033[35m', '\033[0m')
try:
for line in range(head):
print('{}{}{}'.format(yellow, input_.pop(0), default))
if tail:
for line in range(0 - tail, 0):
print('{}{}{}'.format(purple, line, default))
except IndexError:
for line in input_:
rint('{}{}{}'.format(purple, line, default))
Didn't test it.
Now I see that first as well as second loop could cause the IndexError exception.
Quick, how does the real tail
handle an input stream larger than system memory? yes | tail
I don't know what it would do, but reading all of stdin seems like a bad idea.
(Sep-29-2017, 03:03 PM)nilamo Wrote: [ -> ] (Sep-29-2017, 05:16 AM)Skaperen Wrote: [ -> ]off by one: i'm getting 1 more line than specified for the head
Line 16 should be >=
. That's my bad for not really testing that hard.
And, unless otherwise stated, feel free to assume all code I post in the forums is released into the public domain.
now with line 16 changed when i specify 4 i get 3 for head and 3 for tail.
(Sep-29-2017, 08:34 PM)nilamo Wrote: [ -> ]Quick, how does the real tail
handle an input stream larger than system memory? yes | tail
I don't know what it would do, but reading all of stdin seems like a bad idea.
just to be safe, instead of running this on my laptop (not even in a VM on it) i fired up an AWS cloud instance (15GB) and ran i there. it is still running but i am not sure what it is doing. both
yes
and
tail
have used up about 4GB each (seems a lot for
yes
) but are not growing it's not using any swap space, either. my guess is that the real
tail
is counting duplicate lines. if
yes
needs that much space, it must be taken up virtually by the library ... in both processes. there seems to be no slowdown.
Output:
lt1/forums /home/forums 2> ssh ec2-user@2600:1f18:6291:9700:19e2:fac9:a9fb:19d4
Warning: Permanently added '2600:1f18:6291:9700:19e2:fac9:a9fb:19d4' (ECDSA) to the list of known hosts.
__| __|_ )
_| ( / Amazon Linux AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/
[ec2-user@ip-172-31-0-76 ~]$ yes | tail
and in another window and ssh session (not running slow), two commands separated by some time:
Output:
lt1/forums /home/forums 1> ssh ec2-user@2600:1f18:6291:9700:19e2:fac9:a9fb:19d4
Warning: Permanently added '2600:1f18:6291:9700:19e2:fac9:a9fb:19d4' (ECDSA) to the list of known hosts.
Last login: Sun Oct 1 05:43:55 2017 from 2a02:348:61:5d75::f
__| __|_ )
_| ( / Amazon Linux AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-ami/2017.03-release-notes/
[ec2-user@ip-172-31-0-76 ~]$ ps awux|egrep 'yes|tail'|fgrep -v 'grep'
ec2-user 3171 99.9 0.0 4316 708 pts/0 S+ 05:21 24:48 yes
ec2-user 3172 51.2 0.0 4344 732 pts/0 R+ 05:21 12:42 tail
[ec2-user@ip-172-31-0-76 ~]$ ps awux|egrep 'yes|tail'|fgrep -v 'grep'
ec2-user 3171 99.9 0.0 4316 708 pts/0 R+ 05:21 35:21 yes
ec2-user 3172 51.1 0.0 4344 732 pts/0 S+ 05:21 18:06 tail
[ec2-user@ip-172-31-0-76 ~]$
i killed
yes
and
tail
output 10 lines of "y" (as expected).