MySQL替换给定域名的.html链接

3

我的数据库中有许多包含内部链接的内容字段。我需要将链接结构从www.mydomain.de/page.html更改为www.mydomain.de/page/,但替换语句应该考虑域名:

这是预期要替换的内容:

www.mydomain.de/somepage.html -> www.mydomain.de/page/
www.mydomain.de/subfolder/page.html -> www.mydomain.de/subfolder/page/
www.mydomain.de/link.html?param=1 -> www.mydomain.de/page/?param=1
www.mydomain.de/another-link.html#hash -> www.mydomain.de/page/#hash

其他所有链接都应该保持不变,以下是一些示例,但网页上的任何链接都可以是:

 www.some-domain.de/link.html    
 www.another-domain.com/somelink.html

一个内容字段中可以有多个链接:

<p>If you want to read more, click 
<a href="http://www.mydomain.de/page.html">here</a> 
or there <a href="http://www.another-domain.com/somelink.html">there</a>

这是替换操作:
UPDATE tablename 
SET contentfield = REPLACE(contentfield, '.html', '/') 

我的想法(但不知道如何将其陈述):

  • 在前100个字符中是否包含'mydomain.de'
  • '.html'出现的次数和'mydomain.de'出现的次数相同

不必完全匹配所有'mydomain.de'链接,我满意90%,但是不应该在外部链接中进行错误替换。


如果必须完全使用MySQL/ANSI SQL,我认为这是不可能的...除非当然,您现在知道所有可能的不带查询字符串的URL(在这种情况下,它将有些琐碎)。但是如果您不知道...问题在于,您可以使用正则表达式匹配域名(和URL),但您既不能将其提取到某个中间变量/表/任何其他对象中,也不能用正则表达式替换。如果您愿意允许一些外部脚本来完成这项工作,那么这几乎是微不足道的。如果存储过程是一种可能性,那么可能会有一个解决方案,但它可能会很丑陋。 - Jakumi
@JPG 嗯,你可以多次运行更新吗? - Jakumi
@Jakumi 那存储过程可能是一个解决方案,不是吗? - Blank
1
其实,我从来没有写过存储过程,我只是认为它可以完成。我刚刚找到了这个链接:https://dev59.com/13NA5IYBdhLWcg3wX8rk - Jakumi
@Andrea,您是否考虑使用外部UDF库,例如https://github.com/hholzgra/mysql-udf-regexp? - Steve Chambers
4个回答

4

更新:现在已将此内容转为博客文章:http://stevettt.blogspot.co.uk/2018/02/a-mysql-regular-expression-replace.html


请查看以下Rextester Fiddle,我认为应该能够产生您所要求的所有结果:

Rextester Demo

说明

这需要一个模式替换函数,但不幸的是MySQL没有提供这样的功能。因此,我写了一个(基于另一个不太够用的函数),并发布在这里。如引用答案中所述,此函数有一个限制,即不能使用反向引用替换捕获组。因此,在fiddle中稍微调整了一下,以接受进一步的参数,并允许它在找到匹配项后对替换进行递归替换。(注意正则表达式中允许的URL路径字符,参见这个优秀的答案)。

更新SQL

以下SQL将使用下面的函数更新表数据:

UPDATE urls
SET url = reg_replace(
  url,
  'www\\.mydomain\\.de/[-A-Za-z0-9\\._~!\\$&''\\(\\)\\*\\+,;=:@%/]+\\.html',
  '/[^/]+\\.html',
  '/page/',
  TRUE,
  22, -- Min match length = www.mydomain.de/?.html = 22
  0,  -- No max match length
  7,  -- Min sub-match length = /?.html = 7
  0   -- No max sub-match length
  );

函数代码

演示中使用的UDF代码如下所示。注意:该UDF委托给存储过程,因为只有存储过程才允许在MySQL中进行递归

-- ------------------------------------------------------------------------------------
-- USAGE
-- ------------------------------------------------------------------------------------
-- SELECT reg_replace(<subject>,
--                    <pattern>,
--                    <subpattern>,
--                    <replacement>,
--                    <greedy>,
--                    <minMatchLen>,
--                    <maxMatchLen>,
--                    <minSubMatchLen>,
--                    <maxSubMatchLen>);
-- where:
-- <subject> is the string to look in for doing the replacements
-- <pattern> is the regular expression to match against
-- <subpattern> is a regular expression to match against within each
--              portion of text that matches <pattern>
-- <replacement> is the replacement string
-- <greedy> is TRUE for greedy matching or FALSE for non-greedy matching
-- <minMatchLen> specifies the minimum match length
-- <maxMatchLen> specifies the maximum match length
-- <minSubMatchLen> specifies the minimum match length
-- <maxSubMatchLen> specifies the maximum match length
-- (minMatchLen, maxMatchLen, minSubMatchLen and maxSubMatchLen are used to improve
--  efficiency but are optional and can be set to 0 or NULL if not known/required)
-- Example:
-- SELECT reg_replace(txt, '[A-Z0-9]{3}', '[0-9]', '_', TRUE, 3, 3, 1, 1) FROM tbl;
DROP FUNCTION IF EXISTS reg_replace;
DELIMITER //
CREATE FUNCTION reg_replace(subject VARCHAR(21845), pattern VARCHAR(21845),
  subpattern VARCHAR(21845), replacement VARCHAR(21845), greedy BOOLEAN,
  minMatchLen INT, maxMatchLen INT, minSubMatchLen INT, maxSubMatchLen INT)
RETURNS VARCHAR(21845) DETERMINISTIC BEGIN
  DECLARE result VARCHAR(21845);
  CALL reg_replace_worker(
    subject, pattern, subpattern, replacement, greedy, minMatchLen, maxMatchLen,
    minSubMatchLen, maxSubMatchLen, result);
  RETURN result;
END;//
DELIMITER ;

DROP PROCEDURE IF EXISTS reg_replace_worker;
DELIMITER //
CREATE PROCEDURE reg_replace_worker(subject VARCHAR(21845), pattern VARCHAR(21845),
  subpattern VARCHAR(21845), replacement VARCHAR(21845), greedy BOOLEAN,
  minMatchLen INT, maxMatchLen INT, minSubMatchLen INT, maxSubMatchLen INT,
  OUT result VARCHAR(21845))
BEGIN
  DECLARE subStr, usePattern, useRepl VARCHAR(21845);
  DECLARE startPos, prevStartPos, startInc, len, lenInc INT;
  SET @@SESSION.max_sp_recursion_depth = 2;
  IF subject REGEXP pattern THEN
    SET result = '';
    -- Sanitize input parameter values
    SET minMatchLen = IF(minMatchLen < 1, 1, minMatchLen);
    SET maxMatchLen = IF(maxMatchLen < 1 OR maxMatchLen > CHAR_LENGTH(subject),
                         CHAR_LENGTH(subject), maxMatchLen);
    -- Set the pattern to use to match an entire string rather than part of a string
    SET usePattern = IF (LEFT(pattern, 1) = '^', pattern, CONCAT('^', pattern));
    SET usePattern = IF (RIGHT(pattern, 1) = '$', usePattern, CONCAT(usePattern, '$'));
    -- Set start position to 1 if pattern starts with ^ or doesn't end with $.
    IF LEFT(pattern, 1) = '^' OR RIGHT(pattern, 1) <> '$' THEN
      SET startPos = 1, startInc = 1;
    -- Otherwise (i.e. pattern ends with $ but doesn't start with ^): Set start pos
    -- to the min or max match length from the end (depending on "greedy" flag).
    ELSEIF greedy THEN
      SET startPos = CHAR_LENGTH(subject) - maxMatchLen + 1, startInc = 1;
    ELSE
      SET startPos = CHAR_LENGTH(subject) - minMatchLen + 1, startInc = -1;
    END IF;
    WHILE startPos >= 1 AND startPos <= CHAR_LENGTH(subject)
      AND startPos + minMatchLen - 1 <= CHAR_LENGTH(subject)
      AND !(LEFT(pattern, 1) = '^' AND startPos <> 1)
      AND !(RIGHT(pattern, 1) = '$'
            AND startPos + maxMatchLen - 1 < CHAR_LENGTH(subject)) DO
      -- Set start length to maximum if matching greedily or pattern ends with $.
      -- Otherwise set starting length to the minimum match length.
      IF greedy OR RIGHT(pattern, 1) = '$' THEN
        SET len = LEAST(CHAR_LENGTH(subject) - startPos + 1, maxMatchLen), lenInc = -1;
      ELSE
        SET len = minMatchLen, lenInc = 1;
      END IF;
      SET prevStartPos = startPos;
      lenLoop: WHILE len >= 1 AND len <= maxMatchLen
                 AND startPos + len - 1 <= CHAR_LENGTH(subject)
                 AND !(RIGHT(pattern, 1) = '$' 
                       AND startPos + len - 1 <> CHAR_LENGTH(subject)) DO
        SET subStr = SUBSTRING(subject, startPos, len);
        IF subStr REGEXP usePattern THEN
          IF subpattern IS NULL THEN
            SET useRepl = replacement;
          ELSE
            CALL reg_replace_worker(subStr, subpattern, NULL, replacement, greedy, 
                          minSubMatchLen, maxSubMatchLen, NULL, NULL, useRepl);
          END IF;
          SET result = IF(startInc = 1,
                          CONCAT(result, useRepl), CONCAT(useRepl, result));
          SET startPos = startPos + startInc * len;
          LEAVE lenLoop;
        END IF;
        SET len = len + lenInc;
      END WHILE;
      IF (startPos = prevStartPos) THEN
        SET result = IF(startInc = 1, CONCAT(result, SUBSTRING(subject, startPos, 1)),
                        CONCAT(SUBSTRING(subject, startPos, 1), result));
        SET startPos = startPos + startInc;
      END IF;
    END WHILE;
    IF startInc = 1 AND startPos <= CHAR_LENGTH(subject) THEN
      SET result = CONCAT(result, RIGHT(subject, CHAR_LENGTH(subject) + 1 - startPos));
    ELSEIF startInc = -1 AND startPos >= 1 THEN
      SET result = CONCAT(LEFT(subject, startPos), result);
    END IF;
  ELSE
    SET result = subject;
  END IF;
END;//
DELIMITER ;

1
非常感谢您详尽的回答和众多的解释! - Andrea Schmuttermair

2

这个是否符合你的需求?

UPDATE tablename 
    SET contentfield = REPLACE(contentfield, '.html', '/') 
    WHERE contentfield like 'www.mydomain.de/%';

这应该适用于问题中的示例。

如果你愿意,你可以使用条件仅匹配实际包含“.html” 的行。

    WHERE contentfield like 'www.mydomain.de/%.html%'

内容字段可以包含更多的链接 - 我在问题中添加了一个示例。 - Andrea Schmuttermair

1
我只需要将表格导出为CSV格式,然后使用Notepad++/Excel等内部工具将'.html'替换为'/',最后再导入回SQL即可。此外,由于MySQL支持正则表达式,您可以使用它来搜索包含.html的域名。
 mydomain.de[^s]+.html

0

你可以像这样使用

 UPDATE tablename 
 SET contentfield = REPLACE(contentfield, '.html', '/') 
 where contentfield  like 'www.mydomain.de%' 
 AND contentfield  like '%html%'
 AND ( contentfield  not like 'www.another-domain.com/somelink%'  OR  
         contentfield  not like 'www.another-domain.com/subfolder/link%' )

内容字段可以包含更多的链接 - 我在问题中添加了一个示例。 - Andrea Schmuttermair
不幸的是,我知道 "www.mydomain.de" 是一个字符串,而 "another-domain.com" 则是网络上的任何链接。 - Andrea Schmuttermair
你有获取 "another-domain.com" 的逻辑吗?如果有,这个逻辑可以被使用。 - ScaisEdge

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接