Your script is fine, but you need to provide each file separately awk and in reverse order.
$ cat file1.txt
a b c d 100
x y z w 200
p q r s 300
1 2 3 4 400
$ cat file2.txt
. . . . 200
. . . . 400
$ awk 'NR==FNR{a[$5];next} $5 in a {print $0}' file2.txt file1.txt
x y z w 200
1 2 3 4 400
EDIT :
, OP, :
$ awk -F'\t' 'NR==FNR{a[$5];next} $5 in a' <(zcat file2.txt) <(zcat file1.txt)
x y z w 200
1 2 3 4 400
NR - , FNR -
. , NR == FNR
true, awk ( file2.txt).
a[$5] 5- a. awk - , .
,
. next ,
, - awk-.
, : " (file2.txt),
5 a
awk- ".
NR == FNR { a[$5]; next }
, , ,
awk-, (file1.txt ).
$5 in a true, 5-
a. , file1.txt,
, 5- file2.txt.
awk, true,
. , ,
, . ,
$5 in a, awk, file1.txt,
file2.txt, , , .
$5 in a