如何解析HTML以搜索特定内容?

3
给定下面的NSString,它最初是从使用NSXMLParser解析XML文档检索到的CData对象转换而来的。我如何获取书籍的以下属性:标题、书籍封面图像、作者、价格和评分?
这里是我获取上述属性的基本解决方案:
  1. 书名 - 我可能可以通过查看riRssTitle span类来获取这个信息,但然后我就必须找出如何读取ahref url标签之间的标题以获得标题。

  2. 书籍封面 - 我需要通过抓取第一个URL http://ecx.images-amazon.com/images/I/41Lg22K3ViL._SL160_PIsitb-sticker-arrow-dp,TopRight,12,-18_SH30_OU02_.jpg,然后保留http://ecx.images-amazon.com/images/I/41Lg22K3ViL并省略其余部分,然后添加.jpg标记以在稍后检索时获得完整的图像URL。

  3. 书籍作者 - 我必须执行与步骤1相同的步骤,但是要搜索riRssContributor span标记。

  4. 书籍价格 - 这里没有通用的价格标签,但我看到的一个共同点是价格总是在font tag中,然后位于BOLD tag中。

  5. 评分 - 可能可以通过查找包含单词stars的URL来检索,然后获取其后面的数字,4表示4星,任何附加有-5的数字都表示额外的0.5星。因此,3-5表示3.5星。

这样做最好的方法是什么?还有没有快速解析器可以实现我想要做的事情?
这是亚马逊RSS提要的示例:http://www.amazon.co.uk/gp/rss/bestsellers/books/72/ref=zg_bs_72_rsslink
下面是我为每个项目检索到的CData NSString数据。
<div style="float:left;">
    <a class="url" href="http://www.amazon.co.uk/Gone-Girl-Gillian-Flynn/dp/0753827662/ref=pd_zg_rss_ts_b_72_9">
        <img src="http://ecx.images-amazon.com/images/I/41Lg22K3ViL._SL160_PIsitb-sticker-arrow-dp,TopRight,12,-18_SH30_OU02_.jpg" alt="Gone Girl" border="0" hspace="0" vspace="0" />
    </a>
</div>
<span class="riRssTitle">
    <a href="http://www.amazon.co.uk/Gone-Girl-Gillian-Flynn/dp/0753827662/ref=pd_zg_rss_ts_b_72_9">Gone Girl</a>
</span>
<br />
<span class="riRssContributor">
    <a href="http://www.amazon.co.uk/Gillian-Flynn/e/B001JP3W46/ref=ntt_athr_dp_pel_1">Gillian Flynn</a>
    <span class="byLinePipe">(Author)</span>
</span>
<br />
<img src="http://g-ecx.images-amazon.com/images/G/02/x-locale/common/icons/uparrow_green_trans._V192561975_.gif" width="13" align="abstop" alt="Ranking has gone up in the past 24 hours" title="Ranking has gone up in the past 24 hours" height="11" border="0" />
<font color="green">
    <strong></strong>
</font> 674 days in the top 100 
<br />
<img src="http://g-ecx.images-amazon.com/images/G/02/detail/stars-4-0._V192253865_.gif" width="64" height="12" border="0" style="margin: 0; padding: 0;"/>(5704)
<br />
<br />
<a href="http://www.amazon.co.uk/Gone-Girl-Gillian-Flynn/dp/0753827662/ref=pd_zg_rss_ts_b_72_9">Buy new: </a>
<strike>£9.07</strike>
<font color="#990000">
    <b>£3.85</b>
</font>
<br />
<a href="http://www.amazon.co.uk/gp/offer-listing/0753827662/ref=pd_zg_rss_ts_b_72_9?ie=UTF8&condition=all">60 used & new</a> from 
<span class="price">£2.21</span>
<br />
<br />(Visit the 
<a href="http://www.amazon.co.uk/Best-Sellers-Books-Crime-Thrillers-Mystery/zgbs/books/72/ref=pd_zg_rss_ts_b_72_9">Bestsellers in Crime, Thrillers & Mystery</a> list for authoritative information on this product's current rank.)

有人能给我一些指导吗? - Pavan
1
有一天,有人告诉我:https://dev59.com/X3I-5IYBdhLWcg3wq6do#1732454 - Jerome
@JJBoursier 我昨天看到了这个。很有趣的阅读。我的解决方案不需要使用正则表达式来解决。 - Pavan
1
如果是XML,您考虑使用XPath了吗? - Dijkgraaf
你能使用服务器端技术吗?(一个合适的JSON/XML将被发送到设备,但这必须在服务器上完成。) - Salman A
@SalmanA 不好意思,我不能。 - Pavan
4个回答

8

TFHpple绝对是解析HTML的最佳库。在Github上有超过1000个star。

https://github.com/topfunky/hpple

以下是处理该RSS源的obj-c解决方案:

NSString *stringURL = @"http://www.amazon.co.uk/gp/rss/bestsellers/books/72/ref=zg_bs_72_rsslink";
NSURL  *url = [NSURL URLWithString:stringURL];
NSData *htmlData = [NSData dataWithContentsOfURL:url];

TFHpple * doc = [[TFHpple alloc] initWithHTMLData:htmlData];

NSArray *titleElements = [doc searchWithXPathQuery:@"//span[@class='riRssTitle']/a"];
for (TFHppleElement *element in titleElements)
{
    NSString *title = element.firstChild.content;
    NSLog(@"title: %@", title);
}

NSArray *imageElements = [doc searchWithXPathQuery:@"//a[@class='url']/img"];
for (TFHppleElement *element in imageElements)
{
    NSString *image = element.attributes[@"src"];
    NSMutableArray *parts = [[image componentsSeparatedByString:@"/"] mutableCopy];
    NSArray *pathParts = [parts.lastObject componentsSeparatedByString:@"."];
    [parts removeLastObject];
    [parts addObject:[NSString stringWithFormat:@"%@.%@",pathParts.firstObject, pathParts.lastObject]];
    image = [parts componentsJoinedByString:@"/"];
    NSLog(@"image: %@", image);
}

NSArray *authorElements = [doc searchWithXPathQuery:@"//span[@class='riRssContributor']/a"];
for (TFHppleElement *element in authorElements)
{
    NSString *author = element.firstChild.content;
    NSLog(@"author: %@", author);
}

NSArray *priceElements = [doc searchWithXPathQuery:@"//font/b"];
for (TFHppleElement *element in priceElements)
{
    NSString *price = element.firstChild.content;
    NSLog(@"price: %@", price);
}

NSArray *ratingElements = [doc searchWithXPathQuery:@"//img"];
for (TFHppleElement *element in ratingElements)
{
    if (![element.attributes[@"src"] containsString:@"stars"])
        continue;

    NSArray *parts = [element.attributes[@"src"] componentsSeparatedByString:@"-"];
    if (parts.count < 5) continue;

    NSString *rating = [NSString stringWithFormat:@"%@.%@", parts[3], [parts[4] substringToIndex:1]];
    NSLog(@"rating: %@", rating);
}

像你说的一样,你受制于亚马逊的命名规范。


为什么在提供路径的情况下还要使用for循环?如果您正在搜索元素以查找特定内容,我会理解,但是我看到每个for循环的每次迭代都会分配一个值。发生了什么? - Pavan
因为您提供的 RSS 订阅链接有多个项目。for 循环会打印出每个项目中的属性。 - Krys Jurgowski
那么,你只是简单地回显这些项目,并将其作为解决方案提供给我的问题? - Pavan
RSS订阅中有10个项目。每个for循环都会回显你想要解析的6个属性之一。第一个for循环-10本书的标题,第二个for循环-10本书的图片,等等。 - Krys Jurgowski
如果您检查实时数据,您会注意到价格有时会跳动,并且某些值返回null,因为它们在其他标签中跳动。难道没有更好的方法吗?我不能有空值。 - Pavan

4
您可以使用 TFHppleTFHppleElement 来解析上述数据以满足您的需求。
这里是进行此操作的参考链接:点击这里

2
我看到了你在iOS开发者Facebook群组上的帖子,想给你最后一分钟的建议。
由于Amazon没有严格的命名规则,所以你必须搜索整个反馈。这就是我尝试做的事情,但我试图让它看起来不那么像黑客行为。如果你注意到,你会发现有时候反馈会返回缺失的值,如果你试图寻找特定路径名称,所以我也试图弥补这种情况。
为了使其工作,你只需要从以下URL下载NSDictionary类别:https://github.com/nicklockwood/XMLDictionary
.h
#import <Foundation/Foundation.h>

@interface JMAmazonProcessor : NSObject
+(NSArray*)processAmazonResponseWithXMLData:(NSData*)responseObject;
@end

并且针对

.m

#import "JMAmazonProcessor.h"

@implementation JMAmazonProcessor



+(NSString*)getBookTitleWithArray:(NSArray*)array{
    return [[array[0] objectForKey:kAmazonAHREFKey] objectForKey:kAmazonUnderscoreTextKey];
}

+(NSString*)getBookAuthorWithArray:(NSArray*)array{
    id bookAuthor = [[array[1] objectForKey:kAmazonAHREFKey] objectForKey:kAmazonUnderscoreTextKey];

    if(!bookAuthor){
        bookAuthor = [array[1] objectForKey:kAmazonUnderscoreTextKey];
    }

    if([bookAuthor isKindOfClass:[NSArray class]]){
        bookAuthor = [bookAuthor componentsJoinedByString:@" "];
    }

    return bookAuthor;
}
+(NSString*)getPriceFromDictionary:(NSDictionary*)dictionary{
    return [NSString stringWithUTF8String:[[[[dictionary objectForKey:@"font"] lastObject] objectForKey:@"b"] cStringUsingEncoding:NSUTF8StringEncoding]];
}


+(NSString*)getRatingWithCurrentRatingDictionary:(NSDictionary*)ratingDictionary{
    NSString * stars;
    if([ratingDictionary objectForKey:@"_src"]){
        NSString * possibleStarsURL = [ratingDictionary objectForKey:@"_src"];
        if([possibleStarsURL rangeOfString:@"stars-" options:NSCaseInsensitiveSearch].location != NSNotFound){
            stars = [[[[[possibleStarsURL componentsSeparatedByString:@"stars-"] lastObject] componentsSeparatedByString:@"."] firstObject] stringByReplacingOccurrencesOfString:@"-" withString:@"."];
        }
    }


    return stars;

}
+(NSString*)getRatingFromDictionary:(NSDictionary*)dictionary{
    id currentDictionary = [dictionary objectForKey:@"img"];
    NSString *rating;

    if([currentDictionary isKindOfClass:[NSArray class]]){
        for(int i = 0; i < [currentDictionary count]; i++){
            NSDictionary *currentRatingDictionary = [currentDictionary objectAtIndex:i];

            if((rating = [self getRatingWithCurrentRatingDictionary:currentRatingDictionary])){
                break;
            }
        }
    }

    else if([currentDictionary isKindOfClass:[NSDictionary class]]){
        rating = [self getRatingWithCurrentRatingDictionary:currentDictionary];
    }

    if(!rating) rating = @"Rating is not currently available";
    return rating;
}


+(NSArray*)processAmazonResponseWithXMLData:(NSData*)responseObject{
    NSMutableArray *bookEntries = [[NSMutableArray alloc] init];

    NSDictionary * itemDictionary = [[NSDictionary dictionaryWithXMLData:responseObject] objectForKey:kAmazonRootNode];
    for(int i = 0; i < [[itemDictionary objectForKey:kAmazonFeedItemKey] count]; i++){
        RSSBookEntryModel *cBEO = [[RSSBookEntryModel alloc] init];

        NSDictionary *currentItem = [[itemDictionary objectForKey:kAmazonFeedItemKey] objectAtIndex:i];
        NSString *finalXMLString = [NSString stringWithFormat:@"%@%@%@", kAmazonStartTag, [currentItem objectForKey:kAmazonDescriptionKey], kAmazonEndTag];
        NSDictionary *cData = [NSDictionary dictionaryWithXMLString:finalXMLString];

        NSArray *bookDetailsDictionary = [cData objectForKey:kAmazonSpanKey];

        NSString *bIOURL = [[[[cData objectForKey:@"div"] objectForKey:kAmazonAHREFKey] objectForKey:@"img"] objectForKey:@"_src"];
        NSString *bookImageCoverID = [[[[bIOURL componentsSeparatedByString:kAmazonBookCoverBaseURL] lastObject] componentsSeparatedByString:@"."] firstObject];




        cBEO.bookTitle = [self getBookTitleWithArray:bookDetailsDictionary];
        cBEO.bookAuthor = [self getBookAuthorWithArray:bookDetailsDictionary];
        cBEO.bookCoverImageThumbnailURL = [NSString stringWithFormat:@"%@%@%@%@", kAmazonBookCoverBaseURL, bookImageCoverID, kAmazonBookCoverThumbnailSize, kAmazonBookCoverFileExtention];
        cBEO.bookCoverImageOriginalURL = [NSString stringWithFormat:@"%@%@%@%@", kAmazonBookCoverBaseURL, bookImageCoverID, kAmazonBookCoverMaxSize, kAmazonBookCoverFileExtention];
        cBEO.bookPrice = [self getPriceFromDictionary:cData];
        cBEO.bookRating = [self getRatingFromDictionary:cData];

        [bookEntries addObject:cBEO];


    }
    return bookEntries;
}
@end

抱歉。 这就是你想要使用的对象模型,非常简单明了。

@interface RSSBookEntryModel : NSObject

@property (strong, nonatomic) NSString *bookTitle;
@property (strong, nonatomic) NSString *bookAuthor;
@property (strong, nonatomic) NSString *bookCoverImageThumbnailURL;
@property (strong, nonatomic) NSString *bookCoverImageOriginalURL;
@property (strong, nonatomic) NSData *bookCoverThumbnailImage;
@property (strong, nonatomic) NSData *bookCoverOriginalImage;

@property (strong, nonatomic) NSString *bookPrice;
@property (strong, nonatomic) NSString *bookRating;


-(NSString*)description;

@end

这里是我使用的常量,以保持一切整洁。
Constant.h

extern NSString * const kAmazonRootNode;
extern NSString * const kAmazonStartTag;
extern NSString * const kAmazonEndTag;

extern NSString * const kAmazonFeedItemKey;
extern NSString *const kAmazonSpanKey;
extern NSString * const kAmazonDescriptionKey;
extern NSString *const kAmazonUnderscoreTextKey;
extern NSString *const kAmazonAHREFKey;
extern NSString *const kAmazonBookCoverBaseURL;

extern NSString *const kAmazonBookCoverThumbnailSize;
extern NSString *const kAmazonBookCoverMaxSize;
extern NSString *const kAmazonBookCoverFileExtention;

这里是Constants.m文件。

NSString * const kAmazonRootNode = @"channel";
NSString * const kAmazonStartTag = @"<startTag>";
NSString * const kAmazonEndTag = @"</startTag>";

NSString * const kAmazonFeedItemKey = @"item";
NSString *const kAmazonSpanKey = @"span";
NSString * const kAmazonDescriptionKey = @"description";
NSString *const kAmazonUnderscoreTextKey = @"__text";
NSString *const kAmazonAHREFKey = @"a";
NSString *const kAmazonBookCoverBaseURL = @"http://ecx.images-amazon.com/images/";

NSString *const kAmazonBookCoverThumbnailSize = @"._SL100";
NSString *const kAmazonBookCoverMaxSize = @"._SL500";
NSString *const kAmazonBookCoverFileExtention = @".jpg";

哇,好的,我看了下代码,发现你正在处理使用情况的作者代码时,有时返回的值为null。太好了。不过RSSBookEntryModel是什么意思?其中包含了什么内容?还有常量在哪里? - Pavan
我很高兴你注意到了额外的处理 ;) - Jay Maragh
您不需要向数组中输入任何内容,这些方法由主方法使用,并且将Amazon响应对象提供给该主方法。其他所有事情都会被处理,然后您将从该Amazon源接收到您的十个书籍对象数组。 - Jay Maragh
运行得非常好,但这仍然是非常硬编码的。我把它交给你,因为你处理了有时作者值会返回null的情况。谢谢。 - Pavan

1
这里提供的替代方案不是很强,但或许能在某种程度上有所帮助:
//title
console.log("TITLE: " + $(".riRssTitle").text().trim());

//image
console.log("IMAGE: " + $(document).find("img").attr("src"));

//author
console.log("AUTHOR: " + $(".riRssContributor").find("a").text().trim());

//new price and striked price
var new_price_striked_element = $("a:contains('Buy new')").siblings("strike");
if(new_price_striked_element){
   console.log("NEW PRICE STRIKED: " + new_price_striked_element.text().trim());     
}else{
   console.log("NEW PRICE: " + $("a:contains('Buy new')").siblings("b").text().trim()); 
}

//used price
console.log("USED PRICE FROM: " + $(".price").text().trim());

//stars
var url = $("img[src*='stars']").attr("src");
var myRegexp = /stars-([0-9]-[0-9])/g;
var match = myRegexp.exec(url);
console.log("STARS: " + match[1]);

EXAMPLE:http://jsfiddle.net/qpuaxtv3/


亚马逊的商品节点不一致真的很烦人。有时可以检索到价格,而有时则无法检索。不同实体没有特定的节点。最终我只能依赖创建许多不同的情景,并进行大量的“搜索”,这并不是我解决问题的预期方式。 - Pavan
是的,这很弱。它适用于一个特定的场景,但可能不适用于下一个场景。 - carlosHT

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接