MIC 如何批量发产品

写在前

这个世界上解决问题的方法论有很多,然而我认为最有效的还是工程化。

Wikipedia 针对工程化 engineering 的定义如下:
The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.

翻译过来的意思如下:

创造性的应用自然科学的原理设计和开发的服务在经济运行、生命财产安全领域有特定用途的实体或者设计,包括单一的或者组合式的结构、机器、成套设备、生产流程,构建和运行以上设备和流程的设计方案,根据运行状况预测行为模式的模型等的活动统称为工程化。

工程化除了跟自然科学绑定在一起外,更重要的一个概念是成本优化。实现功能或目标的效益要大于付出的成本,这样才能有正收益,工程化才有意义。

如果一件事情机器可以很好的完成任务,则没有必要去使用人工,人工在很多时候都存在单位成本高,效率低下,容易犯错的问题。比如针对 Made In China 产品更新与发布这件事上,人工完全可以被机器取代。

MIC批量发布的工程化方案

  • 产品元数据 (主要对应MIC里面的字段)

    • 基本信息
      • 产品名称 (标题 = 前缀 + 形容词 + 产品关键词 + 后缀)
      • 产品型号
      • 产品关键词
      • 产品图片
    • 产品属性
      • 产品通用属性
      • 产品专用属性
    • 贸易信息 (固定内容)
    • 产品详情
      • 标题
      • 公司简介 (提示相关性)
      • 公司优势 (关怀强化)
      • 产品应用 (连接问题)
      • 详细介绍 (相关性强化 - 解决问题)
      • 包装运输 (增强信任)
      • 生产设备 (增强信任)
      • 质量管控 (增强信任)
      • 资质认证 (增强信任)
      • 联系方式 (转化阶段)
  • 产品发布流程的工程化


MIC是一个基于 web 的 Application, 要完成产品发布,操作需要在浏览器上来完成。产品发布的主要操作为表单的提交( POST方式 )。针对这个操作有很多现成的解决办法,比如 terminal 下的 curl, firefox 的 micros, windows下的火车采集(火车浏览器),Python 自动化测试技术 Selenium,爬虫技术(Scrapy / Scrapy + Splash)。

MIC的登录表单是通过JS的方式提交登录信息,且产品信息填充页面的很多地方都是使用 JS 来验证数据有效性,我是个 JS 菜鸡,因此最终选择了 Selenium 这个解决方案。好处是,我不用考虑 JS 加密与解密的问题,坏处是这种方式操作的效率不高,不过考虑到产品发布这种可以24小时运行对执行时间不敏感的需求来说,Seleminum 够用了,向来我的追求就是能跑就行,还要啥自行车呢。

预警 - Selenium 是需要写代码的,厌恶代码的同学可以绕过了。Talk is cheap,show me the code.


项目的主要结构如下

MIC
+-- data
| + -- products_data.xlsx (产品元数据/基础数据)
+-- images (产品主图)
| + -- images1.jpg
| + -- images2.jpg
| + ...
+-- excel.py ( 将 excel 表格中的数据导入到数据库中,让数据持久化、状态化以方便后期知道任务进行到了哪一步 )
+-- product_node.py ( MIC产品类 - 用于生成需要发布的产品的详细描述 body , 即 WYSJWYG 编辑器需要填充的内容)
+-- mic_publish.py ( 产品发布的操作 - 包括登录账号、设置产品内容、上传图片、添加详细描述、提交发布等 )


以下只针对 product_node.py 和 mic_publish.py 这两处代码进行展示。

产品发布的操作


from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from retry import retry
from product_node import ProductNode
import pymysql
import random
import time

class MicProductPublish:
    #  账户登录信息
    LOGIN_URL = 'https://login.made-in-china.com/sign-in/'
    USER_NAME = 'mic_account_name'
    PASSWORD = 'mic_password'
    # 单个产品的镜像发布数量
    PRODUCT_NODE_MIRRORS = 7
    # 产品模板的链接
    PRODUCT_TEMPLATE_PAGE = 'https://membercenter.made-in-china.com/productmanage.do?xcase=addSimilar&prodId=similar_products_id'
    # 每个产品上传的图片数量
    UPLOAD_IMG_NUMS = 4
    # 可上传的总的图片数量
    PRODUCT_IMG_NUMS = 65
    UNIX_SOCKET = "/Applications/MAMP/tmp/mysql/mysql.sock"
    # 自定义参数属性
    GRADE_SELECT_DICT = {
        '1000': 'FQfmnGaYDEHo',
        '2000': 'CmtxEBGAjnib',
        '3000': 'ynAxmZaDuQHe',
        '4000': 'smfJQhtCuniq',
        '5000': 'CmtEQNGxjJib',
        '6000': 'ZJAnxQfuDEIo',
        '7000': 'sJaQEetSunIo',
        'other': '-1'
    }

    def __init__(self, task_counts):

        self.browser = webdriver.Firefox()
        self.login()

        # 连接数据库
        self.db = pymysql.connect(user='database_user',
                                  password='database_password',
                                  database='database_name',
                                  cursorclass=pymysql.cursors.DictCursor,
                                  unix_socket=self.UNIX_SOCKET)
        # 单次执行发布认为的牌号数量
        self.task_counts = task_counts

    def get_publish_schedule(self):
        # 第一批只操作常见牌号(未发布),获取目标牌号,并限定数量为task_counts
        sql = '''
            SELECT `Grade` FROM `database_name`.`grade` WHERE `Common`=1 AND `Published`=0 LIMIT {task_counts};
        '''.format(task_counts=self.task_counts)

        cursor = self.db.cursor()
        cursor.execute(sql)
        # 返回列表结构 待发布的产品牌号信息,数据结构为{'Grade':'1050'}
        return cursor.fetchall()

    def run_publish_schedule(self):

        # 执行产品发布任务
        # 并对后台的数据库中的状态进行修改
        #
        # 1. 执行单个产品发布任务
        # 2. 修改数据库中此牌号的状态

        for g in self.get_publish_schedule():

            m = self.PRODUCT_NODE_MIRRORS
            # 针对一个牌号发布指定数量的镜像产品
            # 添加 任务出错时,再次尝试, 使用retry包
            while m:
                self.go_to_template_page()
                self.publish_keyword_product_node(g['Grade'])
                m -= 1
            self.update_database(g['Grade'])

    @retry(tries=3, delay=2, max_delay=5)
    def publish_keyword_product_node(self, grade_str):
        self.go_to_template_page()
        self.publish_keyword_product_node_fill_content_and_submit(grade_str)

    def publish_keyword_product_node_fill_content_and_submit(self, grade_str):

        # 这里主要是针对浏览器的操作
        # 关于产品信息的构建,使用 产品详情(PD)类

        # 1. 创建产品类的新实例(牌号)
        product_node = ProductNode(grade_str, self.db)
        # 2.填写产品基本信息
        self.fill_base_info(product_node)
        # 3.上传产品图片
        self.upload_images()
        # 4.填写产品属性
        # - 4.1 颜色/应用/认证/工艺 (这里模板已经确定,不用操作)
        # - 4.2 选择牌号信息
        # - 4.3 回火/表面处理/是否合金/包装运输/规格/商标/原产地/Finish/内包装 (这里模板已经确定,不用操作)
        self.input_properties(product_node)
        # 5.贸易信息(这里模板已经确定,不用操作)
        # 6. 产品详情
        self.fill_body(product_node)
        # 7. 提交发布
        self.browser.implicitly_wait(2)
        self.publish_submit()
        self.browser.implicitly_wait(2)

    # 登录MIC
    def login(self):
        self.browser.get(self.LOGIN_URL)
        user = self.browser.find_element_by_id('logonInfo.logUserName')
        user.send_keys(self.USER_NAME)
        passwd = self.browser.find_element_by_id('logonInfo.logPassword')
        passwd.send_keys(self.PASSWORD)
        submit = self.browser.find_element_by_id('sign-in-submit')
        submit.click()

    # 回到默认模板页面
    def go_to_template_page(self):
        self.browser.get(self.PRODUCT_TEMPLATE_PAGE)
        # 隐式等待,检测页面的载入状态,如完全载入则立即结束等待
        self.browser.implicitly_wait(3)

    def fill_base_info(self, product_node):

        # 填写产品标题
        WebDriverWait(self.browser,20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,'textarea.name-enterarea')))
        product_name = self.browser.find_elements_by_css_selector('textarea.name-enterarea')
        product_name[0].send_keys(product_node.build_product_title())

        # 填写产品型号
        product_model = self.browser.find_element_by_name('prodModel')
        product_model.click()
        product_model.send_keys(product_node.get_keyword_grade()['Grade'] + ' keyword')
        self.browser.implicitly_wait(1)

        # 选择中心词
        self.set_center_word()

        # 填写产品关键词信息
        keywords = product_node.build_product_keyword()
        for k in keywords:
            keyword = self.browser.find_element_by_id('prodKeyword' + str(keywords.index(k)))
            keyword.clear()
            keyword.send_keys(k)

        self.browser.implicitly_wait(1)

    def set_center_word(self):

        WebDriverWait(self.browser,20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,'.J-center-words-box')))
        center_words = self.browser.find_elements_by_css_selector('.J-center-words-box .input-checkbox')
        keyword_center_word = self.browser.find_elements_by_css_selector('.J-center-words-box .input-checkbox[value="keyword"]')

        if keyword_center_word:
            keyword_center_word[0].click()keyword
        else:
            center_words[0].click()

    def upload_images(self):

        # 产品图片板块
        imgs = self.build_images_path()
        css = 'input.upload-selector.J-uploader'
        # 上传图片
        for i in imgs:
            upload = self.browser.find_element_by_css_selector(css)
            self.browser.implicitly_wait(3)
            upload.send_keys(i)

    def build_images_path(self):

        imgs = []

        files_list = list(range(1,self.PRODUCT_IMG_NUMS+1,1))

        for i in random.sample(files_list, self.UPLOAD_IMG_NUMS):
            imgs.append('/MIC/images/image' + str(i) + '.jpg')
        return imgs

    def input_properties(self, product_node):

        # 产品属性板块
        form_select = self.browser.find_elements_by_css_selector('.J-prop-select')

        # 设置牌号
        # 可选值
        grade_select = Select(form_select[2])
        self.set_product_property_grade(product_node.get_keyword_grade()['Series'], grade_select)

        # 设置产品状态
self.browser.find_element_by_id('JindustryPropOtherValue6').send_keys('Mill finish or as your request')

    def fill_body(self, product_node):
        # 切换到编辑器的iframe
        wysiwyg = self.browser.find_element_by_css_selector('.cke_wysiwyg_frame')
        self.browser.switch_to.frame(wysiwyg)
        # 获取主编辑界面
        editor_body = self.browser.find_element_by_css_selector('.cke_editable')
        editor_body.clear()
        # 获取产品信息
        self.browser.switch_to.default_content()
        # 设置产品详细描述
        product_body = product_node.build_product_body()
        # 填充产品详情
        self.browser.execute_script(product_body)
        self.browser.implicitly_wait(2)

        # 切换回页面的主要内容
    def publish_submit(self):

        WebDriverWait(self.browser,10).until(EC.element_to_be_clickable((By.CSS_SELECTOR,'.J-product-submit')))
        btn = self.browser.find_element_by_css_selector('.J-product-submit')
        btn.click()

    def update_database(self, grade_str):

        time_stamp = str(int(time.time()))

        update_sql = 'UPDATE `database_name`.`grade` SET `Published` = 1,  `Published_Date` = "{0}" WHERE `GRADE` = "{1}";'.format(time_stamp, grade_str)
        cursor = self.db.cursor()
        cursor.execute(update_sql)
        # 同步修改到数据库
        self.db.commit()

    def __del__(self):
        self.db.close()
        self.browser.close()

if __name__ == '__main__':
    # 发布20个产品,可以设置更大的值
    MicProductPublish(20).run_publish_schedule()

生成单个产品内容

# 生成产品详情的类
import pymysql
import time
import random

class ProductNode:
    # 类公共变量 - 产品主要详情 - 含有主要字段的placetoken
    PRODUCT_MAIN_BODY = '<br><h2>{product_title}</h2><br><p style=\"color:#666666;text-align:justify;font-size:14px;font-family:Arial, Helvetica, sans-serif;\"><br><img title=\"Company profile of {product_title}\" alt=\" Company profile of {product_title}\"     src=\"//image.made-in-china.com/226f3j00WuqQRGJcaZzh/New-Product-Test-2-for-Image-Name.webp\" srcid=\"275914412\" width=\"1060px\"><br> </p><p style=\"color:#666666;font-size:14px;font-family:Arial, Helvetica, sans-serif;\"><span style=\"font-size:20px;\">Technology driven quality focused keyword & keyword alloy manufacturer</span><br> </p><ul><li><p><span style=\"font-size:16px;\">Henan  keyword Industry Group was founded in 2006, located in Gongyi industry area of Henan Province.</span></p></li><li><p><span style=\"font-size:16px;\">The capacity of our keyword Rolling products has reached 200,000 ton per year, formats available in coil, plates, sheets and foil, material of keyword alloy production line covers 1000-8000 series.</span></p></li><li><p><span style=\"font-size:16px;\">Our stable consistent supply chain, outstanding quality control system and first-rate services have been approved and highly praised by our clients, and built us a good reputation among this industry.</span></p></li><li><p><span style=\"font-size:16px;\">Category of our customers is full of diversity and fall into the zone of transportation, architecture, engineering, aviation, space, electricity and package where our keyword products have been widely used.</span><br><br><br> </p></li></ul><img title=\" company advantages of {product_title}\" alt=\" company advantages of {product_title}\" src=\"//image.made-in-china.com/226f3j00lrkaRudBgKgW/New-Product-Test-2-for-Image-Name.webp\" srcid=\"275914422\" width=\"1060px\"><br><br><br><img title=\"Product Application of {product_title}\" alt=\"Product Application of {product_title}\" src=\"//image.made-in-china.com/226f3j00WucERIAjhKpl/New-Product-Test-2-for-Image-Name.webp\" srcid=\"275914432\" width=\"1060px\"><h3 style=\"color:#0078cc;font-size:20px;font-family:Arial, Helvetica, sans-serif;\">Application</h3><br><div class=\"rich-text-table\"><table style=\"width:1037px;\" cellspacing=\"1\" cellpadding=\"1\" border=\"1\"><tbody><tr><td colspan=\"3\" style=\"text-align:center;width:1028px;\"><br><span style=\"font-size:20px;\">Application of {product_title}</span><br> </td></tr><tr><td style=\"width:292px;\"><ul><li><p><span style=\"font-size:16px;\">Transportation</span></p><ul><li><span style=\"font-size:16px;\">Aircraft & Aerospace Automobile</span></li><li><span style=\"font-size:16px;\">Bus</span></li><li><span style=\"font-size:16px;\">Truck</span></li><li><span style=\"font-size:16px;\">Tank Train</span></li><li><span style=\"font-size:16px;\">Metro Line</span></li><li><span style=\"font-size:16px;\">Ship</span></li><li><span style=\"font-size:16px;\">Boat</span></li><li><span style=\"font-size:16px;\">Yacht</span></li></ul></li></ul></td><td style=\"width:367px;\"><ul><li><p><span style=\"font-size:16px;\">Building & Construction</span></p><ul><li><span style=\"font-size:16px;\">Curtain Wall</span></li><li><span style=\"font-size:16px;\">Roofing Decoration</span></li><li><span style=\"font-size:16px;\">Celling Door</span></li><li><span style=\"font-size:16px;\">Window</span></li><li><span style=\"font-size:16px;\">Floor Framework</span></li><li><span style=\"font-size:16px;\">Structure</span></li></ul></li></ul></td><td style=\"width:354px;\"><ul><li><p><span style=\"font-size:16px;\">Packaging & Container</span></p><ul><li><span style=\"font-size:16px;\">Can</span></li><li><span style=\"font-size:16px;\">Box</span></li><li><span style=\"font-size:16px;\">Case</span></li><li><span style=\"font-size:16px;\">Container Seal</span></li><li><span style=\"font-size:16px;\">Lid Cover</span></li><li><span style=\"font-size:16px;\">Flexible Package</span></li><li><span style=\"font-size:16px;\">Tube Foil</span></li><li><span style=\"font-size:16px;\">Household Foil</span></li></ul></li></ul></td></tr><tr><td style=\"width:292px;\"><ul><li><p><span style=\"font-size:16px;\">Electronics & Appliances</span></p><ul><li><span style=\"font-size:16px;\">Computer</span></li><li><span style=\"font-size:16px;\">Laptop</span></li><li><span style=\"font-size:16px;\">Communication Tools</span></li><li><span style=\"font-size:16px;\">Consumer Electric</span></li><li><span style=\"font-size:16px;\">Heat Exchanger</span></li></ul></li></ul></td><td style=\"width:367px;\"><ul><li><p><span style=\"font-size:16px;\">Machine & Equipment</span></p><ul><li><span style=\"font-size:16px;\">Catering Equipment</span></li><li><span style=\"font-size:16px;\">Textile Machinery</span></li><li><span style=\"font-size:16px;\">Precision Instrument</span></li><li><span style=\"font-size:16px;\">Medical Equipment</span></li></ul></li></ul></td><td style=\"width:354px;\"><ul><li><p><span style=\"font-size:16px;\">Durable Goods</span></p><ul><li><span style=\"font-size:16px;\">Cookware</span></li><li><span style=\"font-size:16px;\">Kitchen Utensils</span></li><li><span style=\"font-size:16px;\">Lamp Cover</span></li><li><span style=\"font-size:16px;\">Air Outlet</span></li><li><span style=\"font-size:16px;\">Light Reflecting Plate</span></li><li><span style=\"font-size:16px;\">Traffic Sign</span></li><li><span style=\"font-size:16px;\">Nameplate</span></li></ul></li></ul></td></tr></tbody></table></div><br><br><br> {keyword_app_info} <h3 style=\"color:#0078cc;font-size:20px;font-family:Arial, Helvetica, sans-serif;\">Element Components</h3><br><br><br><p>Element Components of {product_title}</p><br><div class=\"rich-text-table\">     {element_table} </div><br><br><br><h3 style=\"color:#0078cc;font-size:20px;font-family:Arial, Helvetica, sans-serif;\">Temper</h3><br><br><br><p>Temper of {product_title}</p><br><div class=\"rich-text-table\">     {temper_table} </div><br><br><br><h3 style=\"color:#0078cc;font-size:20px;font-family:Arial, Helvetica, sans-serif;\">Forms</h3><br><br><br><div class=\"rich-text-table\"><p>Common forms of {product_title}</p><br><br>     {forms_list} </div><br><br><br><p>Dimension, tolerance and packing requirement of {product_title} upon request. Please check with our sales to get more information.</p><br><br><br><img title=\"Product package of {product_title}\" alt=\"Product package of {product_title}\" src=\"//image.made-in-china.com/226f3j00SpyQDFtggjzI/New-Product-Test-2-for-Image-Name.webp\" srcid=\"278033782\" width=\"1060px\"><br><br><img title=\"Equipments and Facilities of {product_title}\" alt=\"Equipments and Facilities of {product_title}\" src=\"http://image.made-in-china.com/44f3j00NrPaUWDLaMpn/Product-New-Template-2-Sheet-Plate.jpg\" srcid=\"275914452\" width=\"1060px\"><ul><li><p><span style=\"font-size:16px;\"> keyword Industry has one of one of the most diverse choices of keyword handling equipment in the nation.</span></p></li><li><p><span style=\"font-size:16px;\">We add brand-new equipment to our centers regularly, always striving to use the most recent technology in order to fulfill our the altering requirements of customer.</span></p></li><li><p><span style=\"font-size:16px;\">Our capacity to accomplish high customer complete satisfaction and on-time shipment is largely due to our specialized and educated maintenance personnel and maker drivers.</span></p></li><li><p><span style=\"font-size:16px;\">Our upkeep employees keep our handling, material handling and also delivery tools operating and also offered all the time.</span><br><br> </p></li></ul><br><br><img title=\" Quality Control of {product_title}\" alt=\" Quality Control of {product_title}\" src=\"//image.made-in-china.com/226f3j00VucEUMdaaKpi/New-Product-Test-2-for-Image-Name.webp\" srcid=\"275914462\" width=\"1060px\"><ul><li><p><span style=\"font-size:16px;\"> keyword Industrial is devoted to quality management by providing keyword products and associated services that meet or go beyond client expectations in a sustainable manner while constantly keeping track of and enhancing our products and processes.</span></p></li><li><p><span style=\"font-size:16px;\">Likewise we preserves a high quality of keyword products throughout the entire production process. From the delivery of ingot and smelting to the final dimensional check, substantial attention is provided to all products as outlined by process control checklist for each casting requirement.</span></p></li><li><p><span style=\"font-size:16px;\">We have our own state-art lab and devoted product establishing and evaluation engineers team, a full set test devices including Tensile Tester, Surface Roughness Testers, Direct Reading Emission Spectrometer, Ultrasonic Flaw-detecting Machine, Hardness Testers, Metallurgical Microscope etc.</span></p></li><li><p><span style=\"font-size:16px;\">Our Continuously Hot-rolled keyword Coil Line and Tandem Cold Mills are both import from SMS Group German, the Digital Manufacturing Dispatch Center of our company could fetch real-time operation data feeds directly from the terminal of our production line, which makes us able to locate the problem and do the correction at the very first time. The Quality Assurance system has actually been deeply integrate into every action of our keyword processing flow besides. This is the main reason that we could support our partners from different markets with competitive cost and outstanding quality.</span><br><br> </p></li></ul><img title=\" Certificates of{product_title}\" alt=\" Certificates of {product_title}\" src=\"//image.made-in-china.com/226f3j00iuoQYZJGgjzW/New-Product-Test-2-for-Image-Name.webp\" srcid=\"275914472\" width=\"1060px\"><br><br><br><span style=\"font-size:16px;\">Currently for keyword and keyword alloy sheets and coil, we have passed ISO 9001-2007 Quality Control System Verification. The Certificates displayed here are China Classification Society certificate, DNV Approval of Manufacturer Certificate, CE Certificate for Europe, and SGS Inspection Report.</span><br><br><br><br><br><br><img title=\"Contact sales of {product_title}\" alt=\"Contact sales of {product_title}\" src=\"//image.made-in-china.com/226f3j00hubaYFwzhjri/New-Product-Test-2-for-Image-Name.webp\" srcid=\"275914482\" width=\"1060px\"><br><br><br><br>'

    TITLE_PREFIX = [
        'Best Quality ',
        'Hot Sale ',
        'ASTM JIS EN Standard ',
        'ISO Certificated ',
        'Bottom Price ',
        'Bright Finish ',
        'Rolled ',
        'Top Rated ',
        'Low Price ',
        ]

    TITLE_SUFFIX = [
        ' From Factory',
        ' From Qualified Supplier',
        ' From Audited manufacturer',
        ' Full Size Available',
        ' Fresh Stock',
        ' Factory Direct Sale',
        ' Price Per Ton',
        ' Best Offer Guarantee',
        ]

    PRODUCT_TITLE = ''

    keyword_TEMPERS = {
        '1000': ['F', 'O', 'H'],
        '2000': ['F', 'O', 'W', 'T'],
        '3000': ['F', 'O', 'H'],
        '4000': ['F', 'O', 'W','T'],
        '5000': ['F', 'O', 'H'],
        '6000': ['F', 'O', 'W','T'],
        '7000': ['F', 'O', 'W','T'],
        '8000': ['F', 'O', 'H']
    }

    keyword_TEMPERS_CODES = {
        'F': {
            'defination': 'As fabricated. No special control has been performed to the heat treatment or strain hardening after the shaping process such as casting, hot working, or cold working.',
            'codes': ['F']
        },
        'O': {
            'defination': 'Annealed - This is the lowest strength, highest ductility temper.',
            'codes': ['O'],
        },
        'H': {
            'defination': 'Strain Hardened - (applied to wrought products only) Used for products that have been strengthened by strain hardening, with or without subsequent heat treatment. The designation is followed by two or more numbers as discussed below.',
            'codes': ['H12',
                      'H14',
                      'H16',
                      'H18',
                      'H19',
                      'H111',
                      'H112',
                      'H116',
                      'H21',
                      'H22',
                      'H24',
                      'H26',
                      'H28',
                      'H32',
                      'H321',
                      'H34',
                      'H36',
                      'H38']
        },
        'W': {
            'defination': 'Solution Heat Treated - This is seldom encountered because it is an unstable temper that applies only to alloys that spontaneously age at ambient temperature after heat treatment.',
            'codes': ['W']
        },
        'T': {
            'defination': 'Solution Heat Treated - Used for products that have been strengthened by heat treatment, with or without subsequent strain hardening. The designation is followed by one or more numbers as discussed below.',
            'codes': [
                'T0',
                'T1',
                'T2',
                'T3',
                'T4',
                'T42',
                'T5',
                'T6',
                'T651',
                'T7',
                'T8',
                'T9',
                'T10']
        }
    }

    # 构造函数,类实例化的时候会自动调用该函数
    def __init__(self, keyword_grade, db):
        # 初始化数据库连接
        self.db = db
        self.keyword_grade = keyword_grade
        self.PRODUCT_TITLE = self.build_product_title()

    # 获取产品牌号信息,指针对常用牌号
    def get_keyword__grade(self):
        if self.keyword_grade:
            grade_query_sql = 'SELECT * FROM `database_name`.`grade` WHERE `Grade` = "{0}";'.format(self.keyword_grade)
            cursor = self.db.cursor()
            cursor.execute(grade_query_sql.format(self.keyword_grade))
            return cursor.fetchone()
        else:
            return None

    def get_keyword_temper(self):
        if self.keyword_grade:
            grade = self.get_keyword_grade()
            keyword_temper = self.keyword_TEMPERS[str(grade['Series'])]
            return keyword_temper
        else:
            return False

    # 获取产品特定牌号的应用信息
    def get_keyword_app(self):
        app_query_sql = 'SELECT * FROM `database_name`.`application` WHERE `Grade` = "{0}";'.format(self.keyword_grade)
        cursor = self.db.cursor()
        cursor.execute(app_query_sql.format(self.keyword_grade))
        result = cursor.fetchone()
        if result['App_zh'] == 'nan':
            result['App_zh'] = None
        if result['App_en'] == 'nan':
            result['App_en'] = None
        return result

    def build_keyword_app_info(self):
        app_dict = self.get_keyword_app()

        app_info = '<br> <p>Typical application of {product_title}</p> <br> <p>{app_en}</p> <br> <br> <br>'

        if app_dict['App_en']:
            return app_info.format(product_title=self.PRODUCT_TITLE, app_en = app_dict['App_en'])
        else:
            return ''

    # 获取产品牌号的化学成分信息
    def get_keyword_component(self):
        grade_query_sql = 'SELECT * FROM `database_name`.`component` WHERE `Grade` = "{0}";'.format(self.keyword_grade)
        cursor = self.db.cursor()
        cursor.execute(grade_query_sql)
        return cursor.fetchone()

    def get_keyword_form(self):
        forms = self.keyword_FORMS[str(self.get_keyword_grade()['Series'])]
        return forms

    def build_product_keyword(self):
        keywords = []
        forms = self.get_keyword_form()
        if len(forms) <=3:
            for f in forms:
                keywords.append(' '.join([self.keyword_grade, 'keyword', f]))
        else:
            three_forms = random.sample(forms, 3)
            for g in three_forms:
                keywords.append(' '.join([self.keyword_grade, 'keyword', g]))

        return keywords

    def build_product_title(self):
        prefix = ''.join(random.sample(self.TITLE_PREFIX, 1))
        suffix = ''.join(random.sample(self.TITLE_SUFFIX, 1))
        alloy_str = ''

        if 'A' not in self.keyword_grade:
            alloy_str = ''.join(random.sample(['', 'A', 'AA'], 1))

        forms_str = alloy_str + self.keyword_grade + ' keyword ' + '/'.join(self.get_keyword_form())
        return prefix + forms_str + suffix

    def build_comp_form(self):
        tr_list = self.build_table_tr_list()

        # 构建表格的头部区域
        table = '<table style="width:500px;" cellspacing="1" cellpadding="1" border="1"><thead><tr><td style="width:120px; text-align: center;">' + tr_list[0]['ele'] + '</td><td style="text-align: center;">' + tr_list[0]['value'] +'</td></tr></thead><tbody>'
        for i, tr in enumerate(tr_list):
            if i >= 1:
                if tr['rowspan']:
                    table += '<tr><td style="width:120px; text-align: center">' + tr['ele'] + '</td><td rowspan="2" style="text-align: center;">' + tr['value'] + '</td></tr>'
                elif tr['td'] == '':
                    table += '<tr><td style="width:120px; text-align: center">' + tr['ele'] + '</td></tr>'
                else:
                    table += '<tr><td style="width:120px; text-align: center">' + tr['ele'] + '</td><td style="text-align: center;">' + tr['value'] + '</td></tr>'

        table += '</tbody></table>'

        return table

    def build_table_tr_list(self):
        comp = self.get_keyword_component()

        table_tr_list = []

        for key, value in comp.items():
            if key == 'id':
                pass
            else:
                table_tr_list.append({'ele': key, 'value': value, 'rowspan': False, 'td': 'Default'})

        for i, tr in enumerate(table_tr_list):

            # rowspan 键指示了需要colspan的ta单元格
            # td值为''且不为'Default'时指示了 化学元素的单元格不生成

            if tr['value'] == 'nan':
                table_tr_list[i-1]['rowspan'] = True
                table_tr_list[i]['td'] = ''

        return table_tr_list

    def build_temper_table(self, keyword_temper):
        if isinstance(keyword_temper, list):

            table_body = '<table style="width:1037px;" cellspacing="1" cellpadding="1" border="1"><tbody>'

            for i in keyword_temper:
                table_body += '<tr><td style="width:120px; text-align:center;">' + i + '</td>'
                table_body += '<td><p style="font-size: 12px; font-style:italic; padding: 10px;">' + self.keyword_TEMPERS_CODES[i]['defination'] + '</p>'
                temper_codes_string = ' / '.join(self.keyword_TEMPERS_CODES[i]['codes'])
                table_body += '<p style="font-size: 12px; font-style: italic; padding: 10px;">' + temper_codes_string + '</p></tr>'

            table = table_body + '<tbody></table>'
            return table
        else:
            return ''

    def build_keyword_forms(self):
        forms = self.get_keyword_form()
        forms_output = '<ul>'
        for f in forms:
            forms_output += '<li><p><span style="font-size:16px;">' + f + '</p></li>'
        forms_output += '</ul>'
        return forms_output

    def build_product_body(self):
        return self.PRODUCT_MAIN_BODY.format(
            product_title=self.PRODUCT_TITLE,
            keyword_app_info=self.build_keyword_app_info(),
            element_table = self.build_comp_form(),
            temper_table=self.build_temper_table(self.get_keyword_temper()),
            forms_list=self.build_keyword_forms(),
        )

    def __del__(self):
        pass

跑任务很简单,只需要切换到命令行页面,输入

python3 mic_publish.py

然后浏览器就会自动的为你发产品了。

实测每 2 秒可以发布一个产品,一个小时可以稳定发布 1000+ 产品,偶尔会失败,失败后重试基本都会通过。

发表评论

电子邮件地址不会被公开。 必填项已用*标注