1. h3> scene
The
production environment requires writing to a large number of json files, inserting an attribute in the specified node. As follows: p>
{
"dataSources":{
"test_dataSource_hod":{
"spec":{
"dataSchema":{
"dataSource":"test_dataSource_hod",
"parser":{
"type":"string",
"parseSpec":{
"timestampSpec":{
"column":"timestamp",
"format":"yyyy-MM-dd HH:mm:ss"
},
"dimensionsSpec":{
"dimensions":[
"method",
"key"
]
},
"format":"json"
}
},
"granularitySpec":{
"type":"uniform",
"segmentGranularity":"hour",
"queryGranularity":"none"
},
"metricsSpec":[
{
"name":"count",
"type":"count"
},
{
"name":"call_count",
"type":"longSum",
"fieldName":"call_count"
},
{
"name":"succ_count",
"type":"longSum",
"fieldName":"succ_count"
},
{
"name":"fail_count",
"type":"longSum",
"fieldName":"fail_count"
}
]
},
"ioConfig":{
"type":"realtime"
},
"tuningConfig":{
"type":"realtime",
"maxRowsInMemory":"100000",
"intermediatePersistPeriod":"PT10M",
"windowPeriod":"PT10M"
}
},
"properties":{
"task.partitions":"1",
"task.replicants":"1",
"topicPattern":"test_topic"
}
}
},
"properties":{
"zookeeper.connect":"zookeeper.com:2015",
"druid.discovery.curator.path":"/druid/discovery",
"druid.selectors.indexing.serviceName":"druid/overlord",
"commit.periodMillis":"12500",
"consumer.numThreads":"1",
"kafka.zookeeper.connect":"kafkaka.com:2181,kafka.com:2181,kafka.com:2181",
"kafka.group.id":"test_dataSource_hod_dd"
}
}
needs to be at the end of the properties code> node to add a
"druidBeam. RandomizeTaskId" : "true" code> property.
2. Ideas h3>
the general idea is as follows:
- scan all files to be changed
- confirm the position to be changed
- insert new character
in the file
where I find it a little bit harder is where I confirm the insertion location. We know is "druid. Selectors. Indexing. The serviceName" : "druid/overlord", code> this thing must be in the node, that as long as I can find this stuff, and then in his behind with respect to OK.
ok, we've got the idea, let's write the code.
#!/usr/bin/python
# coding:utf-8
import os
old_string = '"druid/overlord"'
new_string = ('"druid/overlord",' +
'\n ' +
'"druidBeam.randomizeTaskId":"true",')
def insertrandomproperty(file_name):
if '.json' in file_name:
with open(file, 'r') as oldfile:
content = oldfile.read()
checkandinsert(content, file)
else:
pass
def checkandinsert(content, file):
if 'druidBeam.randomizeTaskId' not in content:
# to avoid ^M appear in the new file because of different os
# we replace \r with ''
new_content = content.replace(old_string, new_string).replace('\r', '')
with open(file, 'w') as newfile:
newfile.write(new_content)
else:
pass
if __name__ == '__main__':
files = os.listdir('/home/tranquility/conf/service_bak')
os.chdir('/home/tranquility/conf/service_bak')
for file in files:
insertrandomproperty(file)
is just updating the content in memory, and then rewriting it back to the file. The code is just a rough representation of the idea and can be modified and optimized as needed.
Read More:
- Python SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3:
- Run Python file for the first time with eclipse / pydev: “UTF-8 ‘codec can’t decode byte 0xc4 in position
- Use Python to insert the specified content in the specified line of the specified file
- Pre initialization of list content and length in Python
- python SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: trunca
- The solution of duplicate entry ‘for key’ primary ‘when inserting data in MySQL
- When reading the CSV file with Python 3, the Unicode decodeerror: ‘UTF-8’ codec can’t decode byte 0xd0 in position 0: invalid con appears
- Python modifies word document content and inserts pictures
- [HTML] Python extracts HTML text to TXT
- python reads csv file is an error _csv.Error: iterator should return strings, not bytes (did you open the file in text)
- Parsing the difference between “R” and “RB” patterns of text files (Python)
- [Oracle] when inserting data, “ora-00001: unique constraint” appears
- Several common methods of inserting pictures into latex documents
- How to convert audio to subtitle (text) with Python?
- Endnote inserting documents causes word to crash! ! Solution
- ElementUI implements front-end paging
- Error resolution of unexpected token in JSON at position 0
- Unexpected token u in JSON at position 0
- JS error: unexpected token u in JSON at position 0
- Spring boot thymeleaf crud implements simple functions of adding, deleting, modifying and querying