HTML parsing failure?
This was my first time using oga, and I tried parsing the page https://www.freshsales.io/api/
I am unable to find the divs I am after. I can find them using nokogiri.
Granted the page doesn't validate properly: https://validator.w3.org/nu/?doc=https%3A%2F%2Fwww.freshsales.io%2Fapi%2F
require 'uri'
require 'net/http'
def get(url)
uri = URI(url)
body = Net::HTTP.get(uri)
body.encode('UTF-8', invalid: :replace, undef: :replace, replace: '')
end
page = get('https://www.freshsales.io/api/')
oga = false
if oga
require 'oga'
doc = Oga.parse_html(page)
path='//div[@class="api-url"]'
doc.xpath(path).each do |node|
puts node.text
end
else
require 'nokogiri'
doc = Nokogiri::HTML(page)
path='//div[@class="api-url"]'
doc.xpath(path).each do |node|
puts node.text
end
end
Any idea where the problem lies?
It would be great if there was a section on how to troubleshoot oga.