In this assignment, you will write a Python program somewhat similar to http://www.py4e.com/code3/geoxml.py. The program will prompt for a URL, read the XML data from that URL using urllib and then parse and extract the comment counts from the XML data, compute the sum of the numbers in the file.
We provide two files for this assignment. One is a sample file where we give you the sum for your testing and the other is the actual data you need to process for the assignment.
Sample data: http://py4e-data.dr-chuck.net/comments_42.xml (Sum=2553)Actual data: http://py4e-data.dr-chuck.net/comments_269101.xml (Sum ends with 14)
You do not need to save these files to your folder since your program will read the data directly from the URL. Note: Each student will have a distinct data url for the assignment – so only use your own data url for analysis.
Data Format and Approach
The data consists of a number of names and comment counts in XML as follows:
<comment> <name>Matthias</name> <count>97</count> </comment>
You are to look through all the <comment> tags and find the <count> values sum the numbers. The closest sample code that shows how to parse XML is geoxml.py. But since the nesting of the elements in our data is different than the data we are parsing in that sample code you will have to make real changes to the code.
To make the code a little simpler, you can use an XPath selector string to look through the entire tree of XML for any tag named ‘count’ with the following line of code:
counts = tree.findall('.//count')
Take a look at the Python ElementTree documentation and look for the supported XPath syntax for details. You could also work from the top of the XML down to the comments node and then loop through the child nodes of the comments node.
$ python3 solution.py Enter location: http://py4e-data.dr-chuck.net/comments_42.xml Retrieving http://py4e-data.dr-chuck.net/comments_42.xml Retrieved 4189 characters Count: 50 Sum: 2...
Relevant program given by the teacher (Geoxml.py) :
import urllib.request, urllib.parse, urllib.error import xml.etree.ElementTree as ET import ssl api_key = False # If you have a Google Places API key, enter it here # api_key = 'AIzaSy___IDByT70' # https://developers.google.com/maps/documentation/geocoding/intro if api_key is False: api_key = 42 serviceurl = 'http://py4e-data.dr-chuck.net/xml?' else : serviceurl = 'https://maps.googleapis.com/maps/api/geocode/xml?' # Ignore SSL certificate errors ctx = ssl.create_default_context() ctx.check_hostname = False ctx.verify_mode = ssl.CERT_NONE while True: address = input('Enter location: ') if len(address) < 1: break parms = dict() parms['address'] = address if api_key is not False: parms['key'] = api_key url = serviceurl + urllib.parse.urlencode(parms) print('Retrieving', url) uh = urllib.request.urlopen(url, context=ctx) data = uh.read() print('Retrieved', len(data), 'characters') print(data.decode()) tree = ET.fromstring(data) results = tree.findall('result') lat = results.find('geometry').find('location').find('lat').text lng = results.find('geometry').find('location').find('lng').text location = results.find('formatted_address').text print('lat', lat, 'lng', lng) print(location)
import urllib.request, urllib.parse, urllib.error
import xml.etree.ElementTree as ET
url = ‘http://py4e-data.dr-chuck.net/comments_269101.xml’
uh = urllib.request.urlopen(url)
data = uh.read()
tree = ET.fromstring(data)
counts = tree.findall(‘.//count’)
counts_list = [int(count.text) for count in counts]
What I want to say:
1. The title is very simple, and the iterative expression is very sweet.
2, I mainly records the teacher give the program inside a decode (), and I want to remember the decode and encode () () the difference is the role of decode other encoded string converted to unicode, is the role of encode unicode into other encoded string (https://blog.csdn.net/qq_40134903/article/details/80710882).
200226: In the last class, the teacher talked carefully about decode (which converts a byte into a String) & Encode (convert strings to Bytes) :
String (Unicode)= encode()=> Bytes (UTF-8)= send()=> socket==> network
String (Unicode) < = Bytes ()= Bytes (UTF-8) < =recv()=socket< ==network
- Python 3 urllib has no URLEncode attribute
- Typeerror in Python regular expression: expected string or bytes like object
- Python error: urllib.error.HTTPError : http Error 404: not found
- Crawler overtime error socket.timeout: timed out/NameError: name ‘socket‘ is not defined
- Python: How to parses HTML, extracts data, and generates word documents
- Python parses XML files (parses, updates, writes)
- The Python DOM method iterates over all the XML in a folder
- Decode PUP data Error when reading radar data [How to Solve]
- [Solved]AttributeError: module ‘urllib’ has no attribute ‘quote’
- Python: How to Obtaining Publick IP Quickly
- An introduction to sys modules in Python and how packages are imported and used
- Zlib.error: error – 3 while decompressing: incorrect header check in Python
- [Solved] TypeError: Object of type ‘bytes’ is not JSON serializable
- Python quote error: * * * keyerror: u ‘\ uxx’ [How to Solve]
- Python: How to Solve error While importing windpy
- How to Solve Python ImportError: cannot import name UnrewindableBodyError
- [Solved] Python Read bam File Error: &&OSError: no BGZF EOF marker； file may be truncated
- Normalize error: TypeError: Input tensor should be a float tensor…
- Python raspberry pie starts sending IP address to mailbox
- Parallel processing in Python (Pool.map(), Pool.starmap(), Pool.apply ())