logo标签列表

将JSON转换为垂直表格

jsonbashwidthjqtabular-form
3
我有以下有效载荷，我想生成一个水平列输出，就像下面这样，在条目之间换行。 有人知道如何实现吗？可以直接使用jq或一些有趣的bash方式。
StackId              : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
EventId              : 97b2fbf0-75a3-11ea-bb77-0e8a861a6983
StackName            : cbongiorno-30800-bb-lambda
LogicalResourceId    : cbongiorno-30800-bb-lambda
PhysicalResourceId   : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
ResourceType         : AWS::CloudFormation::Stack
Timestamp            : 2020-04-03T12:06:47.501Z
ResourceStatus       : CREATE_IN_PROGRESS
ResourceStatusReason : User Initiated

EventId              : BBPassword-CREATE_IN_PROGRESS-2020-04-03T12:06:51.336Z
StackName            : cbongiorno-30800-bb-lambda
LogicalResourceId    : BBPassword
PhysicalResourceId   : 
ResourceType         : AWS::SSM::Parameter
Timestamp            : 2020-04-03T12:06:51.336Z
ResourceStatus       : CREATE_IN_PROGRESS

以下是我使用的两个命令来生成输出，但两者都不理想。

  • 我删除了一个通常填充JSON的键，这只会把一切搞砸。
  • 第一个示例插入了一个分隔符，希望以后可以用它来剥离。
  • 第二个示例给出了一个错误xargs：未终止的引号：
  • 在这两种情况下，我都硬编码了格式长度。但对于好奇心的人来说，可以这样做：jq -re '.StackEvents | map(to_entries | map(.key | length) | max) | max'
jq -re '.StackEvents | .[] | del(.ResourceProperties) | . * {"entry":"---"} | to_entries | .[] | "\(.key) \"\(.value?)\""' bin/logs/3.json  | xargs -n 2 printf "%-21s: %s\n"

jq -re '.StackEvents | .[] | del(.ResourceProperties) | . * {"":"\n"} | to_entries | .[] | "\(.key) \"\(.value?)\""' bin/logs/3.json  | xargs -n 2 printf "%-21s: %s\n"

这里是有效载荷：

{
    "StackEvents": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBWebhookLogGroup-CREATE_IN_PROGRESS-2020-04-03T12:06:51.884Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBWebhookLogGroup",
            "PhysicalResourceId": "cbongiorno-30800-bb-lambda",
            "ResourceType": "AWS::Logs::LogGroup",
            "Timestamp": "2020-04-03T12:06:51.884Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceStatusReason": "Resource creation Initiated",
            "ResourceProperties": "{\"RetentionInDays\":\"7\",\"LogGroupName\":\"cbongiorno-30800-bb-lambda\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBUserName-CREATE_IN_PROGRESS-2020-04-03T12:06:51.509Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBUserName",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::SSM::Parameter",
            "Timestamp": "2020-04-03T12:06:51.509Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{\"Type\":\"String\",\"Description\":\"The username for this lambda to operate under\",\"Value\":\"chb0bitbucket\",\"Name\":\"/bb-webhooks/authorization/username\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBWebhookLogGroup-CREATE_IN_PROGRESS-2020-04-03T12:06:51.409Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBWebhookLogGroup",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::Logs::LogGroup",
            "Timestamp": "2020-04-03T12:06:51.409Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{\"RetentionInDays\":\"7\",\"LogGroupName\":\"cbongiorno-30800-bb-lambda\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "BBPassword-CREATE_IN_PROGRESS-2020-04-03T12:06:51.336Z",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "BBPassword",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::SSM::Parameter",
            "Timestamp": "2020-04-03T12:06:51.336Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{\"Type\":\"String\",\"Description\":\"The password for this lambda to operate under with BB. Unfortunately, using an encrypted password is currently not possible\",\"Value\":\"****\",\"Name\":\"/bb-webhooks/authorization/password\"}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "EventId": "97b2fbf0-75a3-11ea-bb77-0e8a861a6983",
            "StackName": "cbongiorno-30800-bb-lambda",
            "LogicalResourceId": "cbongiorno-30800-bb-lambda",
            "PhysicalResourceId": "arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983",
            "ResourceType": "AWS::CloudFormation::Stack",
            "Timestamp": "2020-04-03T12:06:47.501Z",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceStatusReason": "User Initiated"
        }
    ]
}

根据他人的意见，我编写了一个简单的bash脚本，展示了一个小小的异常（列宽不统一）：

#!/usr/bin/env bash
set -e
set -o pipefail

fileCount=$(( $( ls -1 logs/*.json | wc -l) - 1))
for i in $(seq 1 $fileCount); do
    jq -rs '
      def width:      map(keys_unsorted | map(length) | max) | max ;
      def pad($w):    . + (($w-length)*" ") ;

      .[1].StackEvents - .[0].StackEvents | sort_by (.Timestamp) 
      | width as $w | map(to_entries | map("\(.key|pad($w)) : \(.value)"), [""]) 
      | .[][]
    ' "logs/$((i - 1)).json" "logs/$i.json"

done

输出：

StackId              : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
EventId              : ApiKey-CREATE_COMPLETE-2020-04-03T12:07:47.382Z
StackName            : cbongiorno-30800-bb-lambda
LogicalResourceId    : ApiKey
PhysicalResourceId   : KYgzCNAzPw5Tsy3dKBdoTaHlxywijTSrb1d2UIQ2
ResourceType         : AWS::ApiGateway::ApiKey
Timestamp            : 2020-04-03T12:07:47.382Z
ResourceStatus       : CREATE_COMPLETE
ResourceProperties   : {"StageKeys":[{"StageName":"beta","RestApiId":"8n6tijwaib"}]}

StackId            : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
EventId            : bc9371c0-75a3-11ea-b442-1217092af407
StackName          : cbongiorno-30800-bb-lambda
LogicalResourceId  : cbongiorno-30800-bb-lambda
PhysicalResourceId : arn:aws:cloudformation:us-east-1:882038671278:stack/cbongiorno-30800-bb-lambda/97b14e40-75a3-11ea-bb77-0e8a861a6983
ResourceType       : AWS::CloudFormation::Stack
Timestamp          : 2020-04-03T12:07:49.203Z
ResourceStatus     : CREATE_COMPLETE
3个回答
2
JQ没有内置的填充字符串功能，但实现这个功能并不难。在命令行上给定-r/--raw-output选项，下面的脚本将产生您想要的输出。
.StackEvents
| map(del(.ResourceProperties))
| ( [ .[] | keys_unsorted[] ]
    | map(length)
    | max + 1
  ) as $max
| .[]
  | ( keys_unsorted as $keys
    | [ $keys,
      ( $keys
        | map(length)
        | map($max - .)
        | map(. * " " + ": ")
      ),
      map(.)
    ]
    | transpose[]
    | add
  ),
  ""

Online demo

1
这实际上非常聪明，看到这个我竟然错过了在记录之间添加新行的要求。 - Inian
2
这里有一个解决方案，其中包含一些辅助函数，可以推广到其他用途。
def width:      map(keys | map(length) | max) | max ;
def pad($w):    . + (($w-length)*" ") ;

  .StackEvents
| width as $w
| map(del(.ResourceProperties) | to_entries | map("\(.key|pad($w)) : \(.value)"), [""])
| .[][]

如果传递-r给jq，它应该会产生所需的输出。

在线试用！

编辑：正如peak和oguz ismail在评论中指出的那样，这个解决方案可以使用keys_unsorted进行改进，并且应该从宽度计算中排除.ResourceProperties。
下面是经过这些改进的版本：

def width:      map(keys_unsorted | map(length) | max) | max ;
def pad($w):    . + (($w-length)*" ") ;

  .StackEvents
| map(del(.ResourceProperties))
| width as $w
| map(to_entries | map("\(.key|pad($w)) : \(.value)"), [""])
| .[][]

Try it online!（在线尝试！）
2
keys_unsorted会稍微更有效率一些。 - peak
请参考@oguzismail有关在计算宽度时排除ResourceProperties的备注。 - peak
抱歉没有看到这条评论就删除了。但问题是：“如果要删除ResourceProperties，为什么还要考虑它来计算最大宽度？” - oguz ismail
实际上这个已经足够好了，可以重新包含它。但是，我正在流式传输内容，格式不正确。我将更新问题并提供一个脚本来演示。这只是小问题。 - Christian Bongiorno
2
以下是一种解决方案：
  • 使用max/1提高效率

  • 解决了计算Unicode字符串“宽度”的一些问题，例如如果我们要计算“J̲o̲s̲é̲”的“宽度”，则应该计算为4。

请注意，此处定义的jq过滤器grapheme_length忽略控制字符和零宽度空格的问题。

通用功能

def max(stream):
  reduce stream as $x (null; if . == null then $x elif $x > . then $x else . end);

# Grapheme Length ignoring issues with control characters
# Mn = non-spacing mark
# Mc = combining
# Cf = soft-hyphen, bidi control characters, and language tag characters
def grapheme_length:
  gsub("\\p{Mn}";"") | gsub("\\p{Mc}";"") | gsub("\\p{Cf}";"")
  | length;

def pad($w): tostring + (($w - grapheme_length)*" ") ;

主程序

.StackEvents
| max(.[]
      | keys_unsorted[] 
      | select(. != "ResourceProperties") 
      | grapheme_length) as $w
| map(del(.ResourceProperties)
      | to_entries
      | map("\(.key|pad($w)) : \(.value)"), [""])
| .[][]
1
@oguzismail - 已更正。谢谢。 - peak
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接