grep，剪切并从文件中删除 \n

问题描述

我正在处理一个输入文件，该文件在新行中包含用户 ID 列表。在 bash 脚本中，我在该输入文件上运行 while 循环，使用 grep -E 执行 ldapsearch 查询来过滤我想要的结果。生成的输出文件目前格式如下(/mountpoint/out_file_1.out);

uid=user_id1,cn=Users,ou=Department,dc=myORG    
LDAPresource=myORG_RESname1   LDAPresource=myORG_RESname2  
uid=user_id2,dc=myORG  
LDAPresource=myORG_RESname2   LDAPresource=myORG_RESname3

然而，所需的输出应该如下所示；

user_id1;myORG_RESname1
user_id1;myORG_RESname2
user_id2;myORG_RESname2
user_id2;myORG_RESname3

到目前为止，我已经尝试使用 grep 和 cut 来实现上述所需的输出。这是我在上面的第一个结果文件上运行的确切命令：

grep -E '(^uid=|myORG_RESname1|myORG_RESname2|myORG_RESname3)' /mountpoint/out_file_1.out | cut -d,-f1 >&5

产生第二个输出 (/mountpoint/out_file_2.out);

uid=user_id1  
LDAPresource=myORG_RESname1     
LDAPresource=myORG_RESname2  

再次使用 cut 运行另一个 grep：

grep -E 'LDAPresource|uid=' /mountpoint/out_file_2.out | cut -d= -f2 >&6

最终产生这个输出（/mountpoint/out_file_3.out）：

user_id1  
myORG_RESname1  
myORG_RESname2  

这“几乎”是我需要的。我生成的最后一个输出，需要去掉换行符并为找到的每个资源名称重复用户 ID，如已描述的所需输出 (/mountpoint/final_output.out)：

user_id1;myORG_RESname1  
user_id1;myORG_RESname2 

使用：

tr '\n' ';' < input_file > output_file 没有给我想要的结果...

任何想法如何实现？非常感谢任何帮助。

编辑：

这是我正在运行的实际 bash 脚本以供参考：

#!/bin/bash

# assign file descriptor for input fd
exec 3< /mountpoint/userlist
# assign file descriptor for output fd unfiltered
exec 4> /mountpoint/out_file_1.out
# assign file descriptor for output fd filtered
exec 5> /mountpoint/out_file_2.out
# assign file descriptor for output fd final
exec 6> /mountpoint/out_file_3.out

while IFS= read -ru 3 LINE; do
    ldapsearch -h IPADDR -D "uid=admin,dc=myDC" -w somepwd "(uid=$LINE)" LDAPresource >&4
    grep -E '(^uid=|Resource1|Resource2|Resource3)' /mountpoint/out_file_1.out | cut -d,-f1 >&5
    grep -E 'TAMresource|uid=' /mountpoint/out_file_2.out | cut -d= -f2 >&6
    #tr '\n' ';' < input_filename > file
done
# close fd #3 inputfile
exec 3<&-
# close fd #4 & 5 outputfiles
exec 4>&-
exec 5>&-
# exit with 0 success status
exit 0

解决方法

使用您显示的示例，请尝试以下操作。使用 GNU awk 中的示例编写和测试。

awk '
match($0,/uid=[^,]*/){
  val1=substr($0,RSTART+4,RLENGTH-4)
  next
}
{
  val=""
  while($0){
    match($0,/LDAPresource=[^ ]*/)
    val=(val?val OFS:"")(val1 ";" substr($0,RSTART+13,RLENGTH-13))
    $0=substr($0,RSTART+RLENGTH)
  }
  print val
}' Input_file

说明：为以上添加详细说明。

awk '                                 ##Starting awk program from here.
match($0,]*/){                ##Using match function to match regex uid= till comma comes in current line.
  val1=substr($0,RLENGTH-4)  ##Creating val1 variable which has sub string of matched regex of above.
  next                                ##next will skip all further statements from here.
}
{
  val=""                              ##Nullifying val variable here.
  while($0){                          ##Running loop till current line value is not null.
    match($0,/LDAPresource=[^ ]*/)    ##using match to match regex from string LDAPresource= till space comes.
    val=(val?val OFS:"")(val1 ";" substr($0,RLENGTH-13))  ##Creating val which has val1 ; and sub string of above matched regex.
    $0=substr($0,RSTART+RLENGTH)      ##Saving rest of line in current line.
  }
  print val                           ##Printing val here.
}' Input_file                         ##Mentioning Input_file name here.

,

您要执行的转换的规范不清楚。看来您想成对处理行，取每对第一行表示的 uid 属性和每对第二行指定的两个 LDAPresource 属性，并将它们组合成两行，每行包含id;resource 对。

首先，我不会为此使用 grep 或 cut。 sed 或 awk 会是更合适的工具。我更像是一个 sed 人而不是一个 awk 人，但我确信一个非常简单的 awk 脚本可以一次性完成这项工作。对于 sed，我会使用两个：

• 首先，从您的输入到第三个输出，如下所示：

  sed 's/^[^=]*=//; s/,.*//; n; s/LDAPresource=//g; s/ \{1,\}/\n/'
  

• 其次，将生成的三元行组合起来以实现您想要的输出：

  sed 's/$/;/; h; N; x; N; H; x; s/;\n/;/g'
  

您可以将它们组合成一个命令（尽管我肯定会建议为此编写一个脚本，而不是在命令行中输入所有内容）：

sed 's/^[^=]*=//; s/,\}/\n/' /mountpoint/out_file_1.out |
  sed 's/$/;/; h; N; x; N; H; x; s/;\n/;/g'

说明

给定的每个 sed 命令指定一个以分号分隔的步骤序列，在一个循环中执行，直到输入用完为止。

这是多行形式的第一个，带有注释

# The next line of input is implicitly read into sed's pattern space,sans trailing newline

# Replace the leading substring up to the first '=' with nothing (that is,delete it)
s/^[^=]*=//

# Replace the substring from the first comma to the end of the line with nothing.
# This leaves just the uid value.
s/,.*//

# Print the contents of the pattern space followed by a newline (supposes that the
# -n command line option has not been given) and replace the contents of the pattern
# space with the next line of input.
n

# Replace all substrings 'LDAPresource=' in the pattern space with nothing
s/LDAPresource=//g

# Replace the first (and only) run of one or more consecutive space characters with a newline
s/ \{1,\}/\n/

# The remaining contents of the pattern space and a trailing newline are printed at this point
# (assuming no '-n' option) and the cycle repeats.

第二个是：

# The next line of input is implicitly read into sed's pattern space sans trailing newline

# Substitute a semicolon (;) for the zero-length space at the end of the line (that
# is,append a semicolon).
s/$/;/

# Copy the contents of the pattern space into the hold space.  Both spaces then contain
# the uid plus a semicolon
h

# Append a newline followed by the next line of input (sans trailing newline) to the
# pattern space
N

# Swap the contents of the pattern and hold spaces.
x

# Append a newline followed by the next line of input (sans trailing newline) to the
# pattern space
N

# Append a newline followed by the contents of the pattern space to the hold space.
# After this,the contents of the hold space have the form
# <uid>;<newline><resource1><newline><uid>;<newline><resource2>
H

# Swap the pattern and hold spaces
x

# Replace each (semicolon,newline) pair with just a semicolon.  This completes
# joining the uid and resource pairs into semicolon-(only-)delimited form,# leaving a newline between each pair
s/;\n/;/g

# The remaining contents of the pattern space and a trailing newline are printed at this
# point (assuming no '-n' option) and the cycle repeats.

,

$ awk -F'[=,[:space:]]+' -v OFS=',' 'NR%2{uid=$2; next} {print uid,$2 ORS uid,$4}' file
user_id1,myORG_RESname1
user_id1,myORG_RESname2
user_id2,myORG_RESname3