百度智能运维工程架构

背景:为什么要做智能运维

 

百度云智能运维团队在运维工具和平台研发方向历史悠久,支撑了全百度数十万规模的服务器上的运维服务,所提供的服务包括服务管理、资源定位、监控、部署、分布式任务调度等等。最近几年,团队着力于发展智能化运维能力以及AIOps产品化建设。

 

众所周知,百度除了搜索业务之外,还有很多其他的业务线,有像地图、百科、知道、网盘这样的老牌业务,也有诸如像教育、医疗这样的新兴业务,每个业务在规模上、服务架构上都有很大差异。业务本身对稳定性的要求很高,需要保持99.995%的高可用,同时在业务上云的背景下,虚拟化、混合云等都给我们带来了新的挑战。

 

百度运维经历了从脚本&工具、基础运维平台、开放可定制运维平台到我们现在的智能运维平台,这样四个阶段的转变。过去运维的核心目标是提升效果,比如持续交付的速度、服务稳定性、运营成本等。经过这么多年的建设,整个运维行业已经非常成熟,而我们所支撑业务规模仍在不断增长,越来越多的运维场景和问题无法用传统方法来解决,而运维效率也难以继续支撑业务规模的快速扩张,所以我们更加关注怎么样解放运维自身的效率,以及解决传统运维方法(人工、自动化)所解决不了的问题。

 

这就好比从马车到汽车是为了提升运输效率,而到汽车已经接近饱和的时候,我们又希望用自动驾驶把驾驶员从开车这项体力劳动中解放出来,不仅可以增加运行效率,同时也可以减少交通事故率,这也是我们对智能运维的诉求。

 

发展:AIOps,从理念到落地

 

2016年Gartner报告中提出了AIOps概念,也就是Algorithmic IT Operations;基于算法的IT运维,主要指用大数据、机器学习驱动自动化、服务台、监控这些场景下的能力提升。

 

我们从2014年开始做智能运维方面的探索,最开始也是集中在监控指标分析、报警分析、故障根因分析、性能和成本分析这些方面,到2016年我们已经完成将AI应用于完整的运维平台研发的论证。在我们语义下的AIOps,目标是将人的知识和运维经验与大数据、机器学习技术相结合,开发成一系列的智能策略,融入到运维系统中。用这样的智能运维系统去完成运维任务,是我们所认为的AIOps,也就是Artificial Intelligence IT Operations。有意思的是,2017年之后的Gartner报告也将AIOps的概念改成了Artificial Intelligence IT Operations。

 

我们认为AIOps中有三部分不可或缺,一个是运维开发框架,这个是我们后续智能运维研发的骨架,第二个是运维知识库,这是让骨架能与我们真实线上环境关联起来的关键因素,起到了血肉的作用,让骨架能动起来。而最后一个则是运维策略库,这是运维的大脑,控制着运维平台的行为。

 

使用运维开发框架实现的运维程序,我们称其为运维机器人。运维机器人可以在多种不同的运维场景下提供多样的运维能力,服务不同类型的业务和用户。

 

框架:新的运维开发模式

 

运维开发框架基于这样一个抽象,就是如果我们把线上环境看做一个黑盒服务,那么我们对它的操作无非读写两类,所谓的写也就是操作控制流,是那种要对线上状态做一些改变的操作,我们常说的部署、执行命令,都属于这一类;另一类是读,指的是数据流,也就是要从线上获取状态数据,并进行一些聚合统计之类的处理,我们常说的指标汇聚、异常检测、报警都在这个里面。通过运维知识库,可以在这两种操作的基础上,封装出多种不同的运维机器人,对业务提供高效率、高质量以及高可用方面的能力。

 

根据操作流和数据流的不同,我们把框架分成了两部分,最基础的是运维执行框架,在这之上,加上分布式计算组件的支持,我们还建设了用于运维大数据计算的计算框架。

 

1工程化

 

运维开发框架给开发者提供一系列的开发套件,除了包含了一系列的基础能力,还包含了一个标准的运维工程研发流程。

 

在过去,运维研发采用简单的开发-使用方式,缺少必要的测试维护。而现在,在代码开发阶段,可以通过执行框架,用统一的操作接口库提升研发效率。在测试阶段,开发套件提供了单测和仿真系统,简化测试环境搭建。在上线后的阶段,通过状态服务和托管系统,可满足在各灾难场景下的运维机器人的自维护。

 

2组件化

 

运维开发框架通过三种不同的组件功能组合成运维机器人。分别是感知器、决策器和执行器。这三种组件针对各自使用场景,提供了多种架构能力。

 

感知器运维机器人的眼睛和耳朵感,就像人有两个眼睛和两个耳朵一样。运维机器人也可以挂载多个感知器来获取不同事件源的消息,比如监控的指标数据或者是报警事件,变更事件这些,甚至可以是一个定时器。这些消息可以以推拉两种方式被感知器获取到。这些消息也可以做一定的聚合,达到阈值再触发后续处理。

 

决策器是运维机器人的大脑,所以为了保证决策的唯一,机器人有且只能有一个决策器。决策器也是使用者主要要扩展实现的部分。除了常见的逻辑判断规则之外,未来我们还会加入决策树等模型,让运维机器人自主控制决策路径。

 

执行器是运维机器人的手脚,所以同样的,执行器可以并行的执行多个不同的任务。执行器将运维长流程抽象成状态机和工作流两种模式。这样框架就可以记住当前的执行状态,如果运维机器人发生了故障迁移,还可以按照已经执行的状态让长流程断点续起。

 

知识库:运维的知识图谱

 

知识库是智能运维架构中非常重要的一部分:所有要处理的数据都来自知识库,以及所有处理后的数据也都会再进入到知识库中。知识库由三部分组成,分别是元数据、状态数据和事件数据。持续的数据建设,是智能运维建设的关键。

 

考虑到未来需要对接不同的内部云平台和公有云平台,所以我们的运维数据也需要从底层的多种不同的运维平台中抽取,清洗和做数据的整合。并以尽可能高的时效性提供给平台用户使用。因此我们知识库建设遵照这四个能力指标进行,分别是全、准、新、稳。

 

由于知识库涉及的存储的内容篇幅太大,并且是相对独立的一块工作,所以这里就不再展开了。

 

实践:运维机器人

 

单机房故障自愈是2017年我们完成的重点项目,目标是将单机房范围的故障自愈水平普遍提升到L4级(整个处理过程,包括决策过程基本无人介入)。当然,另一部分原因是过去一两年发生的几次业界重大线上事故,我们希望可以防微杜渐,进一步提升MTTR水平。

 

相比较原有的单机房故障处理方式,在感知、决策、执行三个方面,L4级的单机房故障自愈系统效果显著:

 

1.感知方面,智能异常检测算法替代过去大量误报漏报的阈值检测方法;

 

2.决策方面,具备全局信息、自动决策的算法组件替代了过去“老中医会诊”的人工决策模式;

 

3.执行方面,状态机等执行长流程组件的加入,让执行过程可定位、可复用。

 

目前L4级的单机房故障自愈,已经覆盖百度大多数核心业务线,止损效率可做到分钟级,最快秒级止损,较人工止损效率提升60%-99%。

 

总结

 

随着AIOps逐渐走向成熟和产品化,必将有越来越多的运维场景被AIOps所变革,而我们,百度云智能运维团队,也希望秉承着这个方向,为行业贡献更多的创新理念、技术和产品,欢迎大家一起加入探讨。

 

最后,用一句话来总结下工程架构对于智能运维的意义:

 

框架在手,AI我有:智能时代,框架会越来越重要,从机器学习框架TensorFlow到自动驾驶框架Apollo,概莫能外。

 

收藏 评论(1)
分享到:
共1条回复 最后由Q1058204131 回复于2019-08-05 15:27
#2 Q1058204131 回复于2019-08-05

不错的转发下,有啥东东,抱歉
http://gbsgjd.wikidot.com/,
http://mfghdg.wikidot.com/,
http://jsdfghd.wikidot.com/,
http://hfsgf.wikidot.com/,
http://fcnfgsg.wikidot.com/,
http://jfjdsgf.wikidot.com/,
http://hkdtrj.wikidot.com/,
http://fsdgshk.wikidot.com/,
http://fhsdgj.wikidot.com/,
http://fgsdfgs.wikidot.com/,
http://mczfjkm.wikidot.com/,
http://nzgsgj.wikidot.com/,
http://fhsahh.wikidot.com/,
http://ghsdhd.wikidot.com/,
http://nfdgsdd.wikidot.com/,
http://mfgxjch.wikidot.com/,
http://gfhfsa.wikidot.com/,
http://goyfhglk.wikidot.com/,
http://fkgkjjk.wikidot.com/,
http://hgdhds.wikidot.com/,
http://mhdfhs.wikidot.com/,
http://yursdff.wikidot.com/,
http://hdfgdgs.wikidot.com/,
http://shanghhfg.wikidot.com/,
http://fhsshbx.wikidot.com/,
http://mngbsd.wikidot.com/,
http://jsrebx.wikidot.com/,
http://kdgszg.wikidot.com/,
http://hdfhsa.wikidot.com/,
http://jfdsfhd.wikidot.com/,
http://nfgssnt.wikidot.com/,
http://fdgsdas.wikidot.com/,
http://hjrszd.wikidot.com/,
http://hgjskk.wikidot.com/,
http://hedjszm.wikidot.com/,
http://hjuskf.wikidot.com/,
http://kdsrjsg.wikidot.com/,
http://nfdgsar.wikidot.com/,
http://ghkddsj.wikidot.com/,
http://fgjcbxg.wikidot.com/,
http://tgsahd.wikidot.com/,
http://zgsgsrk.wikidot.com/,
http://qrysxk.wikidot.com/,
http://gstrho.wikidot.com/,
http://hckdsk.wikidot.com/,
http://shafhf.wikidot.com/,
http://zsfxmnh.wikidot.com/,
http://mzjtia.wikidot.com/,
http://nfdhssg.wikidot.com/,
http://hkffbd.wikidot.com/,
http://zdfwjt.wikidot.com/,
http://xcrghs.wikidot.com/,
http://nsgzfg.wikidot.com/,
http://hgsdsa.wikidot.com/,
http://aezfgh.wikidot.com/,
http://bxbned.wikidot.com/,
http://gabzjjg.wikidot.com/,
http://hsgyfj.wikidot.com/,
http://hsfdhf.wikidot.com/,
http://zvshetg.wikidot.com/,
http://zzyurej.wikidot.com/,
http://lchdhjw.wikidot.com/,
http://hzjstgrs.wikidot.com/,
http://dzjhkfy.wikidot.com/,
http://jndhas.wikidot.com/,
http://zbhfher.wikidot.com/,
http://ytdghs.wikidot.com/,
http://whhsvs.wikidot.com/,
http://wfkfdsz.wikidot.com/,
http://qdjsgt.wikidot.com/,
http://mgssgtg.wikidot.com/,
http://jdgsjah.wikidot.com/,
http://fhkfak.wikidot.com/,
http://hfhfdds.wikidot.com/,
http://hskefx.wikidot.com/,
http://xykjdg.wikidot.com/,
http://qankfhd.wikidot.com/,
http://xakfhd.wikidot.com/,
http://bjsgsw.wikidot.com/,
http://hzgdstg.wikidot.com/,
http://hfgfhn.wikidot.com/,
http://ashdfyh.wikidot.com/,
http://dldhds.wikidot.com/,
http://wsjfh.wikidot.com/,
http://bxkdhkf.wikidot.com/,
http://ykkfpgh.wikidot.com/,
http://jzkfphj.wikidot.com/,
http://pjkfpof.wikidot.com/,
http://hdkdjf.wikidot.com/,
http://ddkfpgj.wikidot.com/,
http://cykfpjf.wikidot.com/,
http://hldkfpt.wikidot.com/,
http://jygkfphj.wikidot.com/,
http://lzkfphj.wikidot.com/,
http://tlkfphk.wikidot.com/,
http://jqkfpjh.wikidot.com/,
http://tskfpjg.wikidot.com/,
http://xzkfojh.wikidot.com/,
http://wxkfpjh.wikidot.com/,
http://zykfpjh.wikidot.com/,
http://yckfpkjn.wikidot.com/,
http://ntkfpjh.wikidot.com/,
http://szkfpjh.wikidot.com/,
http://lygkfphj.wikidot.com/,
http://chgeesd.wikidot.com/,
http://cckfplk.wikidot.com/,
http://jlkfplk.wikidot.com/,
http://thkfplk.wikidot.com/,
http://spkfplk.wikidot.com/,
http://yzkfplk.wikidot.com/,
http://bckfplk.wikidot.com/,
http://ddjkfplk.wikidot.com/,
http://jjhekfplk.wikidot.com/,
http://henkfplk.wikidot.com/,
http://jnskfplk.wikidot.com/,
http://dqkfplk.wikidot.com/,
http://gykfplk.wikidot.com/,
http://qthkfplk.wikidot.com/,
http://lpskfplk.wikidot.com/,
http://zykfplk.wikidot.com/,
http://kmkfplk.wikidot.com/,
http://ljkfplk.wikidot.com/,
http://qjkfplk.wikidot.com/,
http://yxkfplk.wikidot.com/,
http://zygkfplk.wikidot.com/,
http://zzkfplk.wikidot.com/,
http://qzkfplk.wikidot.com/,
http://xmkfplk.wikidot.com/,
http://ptkfplk.wikidot.com/,
http://fzkfplk.wikidot.com/,
http://tjkfplk.wikidot.com/,
http://cskfplk.wikidot.com/,
http://sjzkfplk.wikidot.com/,
http://cqkfplk.wikidot.com/,
http://bjkfplk.wikidot.com/,
http://dfsgfda.wikidot.com/,
http://dsfgdsf.wikidot.com/,
http://dsfgdsg.wikidot.com/,
http://sdgdsfyh.wikidot.com/,
http://dsgdhg.wikidot.com/,
http://ljmdsgdfg.wikidot.com/,
http://fdhggfkj.wikidot.com/,
http://fhfdsf.wikidot.com/,
http://dsfgsdhy.wikidot.com/,
http://dsfgdfftg.wikidot.com/,
http://sdtgdfd.wikidot.com/,
http://dsgdszfg.wikidot.com/,
http://jhtgjh.wikidot.com/,
http://zdfgds.wikidot.com/,
http://dsgdfdcx.wikidot.com/,
http://vdsghg.wikidot.com/,
http://aghfghj.wikidot.com/,
http://dsgdsdfg.wikidot.com/,
http://dfgsfg.wikidot.com/,
http://dfggd.wikidot.com/,
http://dfjhklfg.wikidot.com/,
http://gfvjhfg.wikidot.com/,
http://dsfgfdf.wikidot.com/,
http://fdstdy.wikidot.com/,
http://dsftdrcg.wikidot.com/,
http://bgfcgh.wikidot.com/,
http://ghfhj.wikidot.com/,
http://hgfghv.wikidot.com/,
http://sdgf.wikidot.com/,
http://gjmhfgh.wikidot.com/,
http://vhgvh.wikidot.com/,
http://hfghg.wikidot.com/,
http://fdgddf.wikidot.com/,
http://sdgd.wikidot.com/,
http://dsfdf.wikidot.com/,
http://sdgfd.wikidot.com/,
http://dsgdf.wikidot.com/,
http://fgdfsh.wikidot.com/,
http://fgsdf.wikidot.com/,
http://dsfgdf.wikidot.com/,
http://sdgfd.wikidot.com/main:layout,
http://sdgfd.wikidot.com/system:join,
http://sdgfd.wikidot.com/help:_home,
http://sdgfd.wikidot.com/main:contact,
http://sdgfd.wikidot.com/help:first-time-user,
http://sdgfd.wikidot.com/help:creating-pages,
http://sdgfd.wikidot.com/help:editing-pages,
http://sdgfd.wikidot.com/help:navigation-bars,
http://sdgfd.wikidot.com/help:using-modules,
http://sdgfd.wikidot.com/help:templates,
http://dsgdf.wikidot.com/main:layout,
http://dsgdf.wikidot.com/system:join,
http://dsgdf.wikidot.com/help:_home,
http://dsgdf.wikidot.com/main:contact,
http://dsgdf.wikidot.com/help:first-time-user,
http://dsgdf.wikidot.com/help:quick-reference,
http://dsgdf.wikidot.com/help:creating-pages,
http://dsgdf.wikidot.com/help:editing-pages,
http://dsgdf.wikidot.com/help:navigation-bars,
http://dsgdf.wikidot.com/help:templates,
http://dsgdf.wikidot.com/help:css-themes,
http://fgdfsh.wikidot.com/main:layout,
http://fgdfsh.wikidot.com/system:join,
http://fgdfsh.wikidot.com/main:about,
http://dsgdf.wikidot.com/main:about,
http://sdgfd.wikidot.com/main:about,
http://fgsdf.wikidot.com/main:about,
http://fgsdf.wikidot.com/main:layout,
http://fgsdf.wikidot.com/system:join,
http://fgsdf.wikidot.com/help:_home,
http://fgsdf.wikidot.com/main:contact,
http://fgsdf.wikidot.com/help:first-time-user,
http://fgsdf.wikidot.com/help:quick-reference,
http://fgsdf.wikidot.com/help:creating-pages,
http://fgsdf.wikidot.com/help:editing-pages,
http://fgsdf.wikidot.com/help:navigation-bars,
http://fgsdf.wikidot.com/help:templates,
http://fgsdf.wikidot.com/help:css-themes,
http://dsfgdf.wikidot.com/main:about,
http://dsfgdf.wikidot.com/main:layout,
http://dsfgdf.wikidot.com/system:join,
http://dsfgdf.wikidot.com/help:_home,
http://dsfgdf.wikidot.com/main:contact,
http://dsfgdf.wikidot.com/help:first-time-user,
http://dsfgdf.wikidot.com/help:quick-reference,
http://dsfgdf.wikidot.com/help:creating-pages,
http://dsfgdf.wikidot.com/help:editing-pages,
http://dsfgdf.wikidot.com/help:navigation-bars,
http://dsfgdf.wikidot.com/help:templates,
http://dsfgdf.wikidot.com/help:css-themes,
http://fgsdf.wikidot.com/main:home,
http://bcvfdf.wikidot.com/,
http://fdsgfxd.wikidot.com/,
http://lmhjvb.wikidot.com/,
http://dsfsdfg.wikidot.com/,
http://mfcgdsfg.wikidot.com/,
http://mhhdf.wikidot.com/,
http://ljkhjc.wikidot.com/,
http://wefdslo.wikidot.com/,
http://iuydfh.wikidot.com/,
http://pfyfdf.wikidot.com/,
http://ertryrt.wikidot.com/,
http://werdsfsx.wikidot.com/,
http://puhjigh.wikidot.com/,
http://iyufh.wikidot.com/,
http://iytrc.wikidot.com/,
http://uysedhb.wikidot.com/,
http://opirfgh.wikidot.com/,
http://uyrthf.wikidot.com/,
http://ytydfgc.wikidot.com/,
http://retgvcx.wikidot.com/,
http://yuifdg.wikidot.com/,
http://qwreersd.wikidot.com/,
http://tyutgf.wikidot.com/,
http://resfdsf.wikidot.com/,
http://rteddf.wikidot.com/,
http://aqddcz.wikidot.com/,
http://ohjvf.wikidot.com/,
http://yursdf.wikidot.com/,
http://wzsdasd.wikidot.com/,
http://phgfdc.wikidot.com/,
http://sdfwew.wikidot.com/,
http://dtfgrerf.wikidot.com/,
http://reweas.wikidot.com/,
http://qwerad.wikidot.com/,
http://qasdfd.wikidot.com/,
http://dfsgdf.wikidot.com/,
http://fddsfgdf.wikidot.com/,
http://jhiujyu.wikidot.com/,
http://sgtyrt.wikidot.com/,
http://asdsz.wikidot.com/,
http://srfsdf.wikidot.com/,
http://desrs.wikidot.com/,
http://sdfser.wikidot.com/,
http://sddfgs.wikidot.com/,
http://sdfsdg.wikidot.com/,
http://dsfsgfe.wikidot.com/,
http://gwserg.wikidot.com/,
http://xcfdg.wikidot.com/,
http://klfghgf.wikidot.com/,
http://gfdaa.wikidot.com/,
http://yedrcg.wikidot.com/,
http://swefzf.wikidot.com/,
http://dsfsza.wikidot.com/,
http://swezd.wikidot.com/,
http://hghgshg.wikidot.com/,
http://rtsdf.wikidot.com/,
http://dsaew.wikidot.com/,
http://dsfrwlo.wikidot.com/,
http://srdtg.wikidot.com/,
http://dfgwews.wikidot.com/,
http://dsggrw.wikidot.com/,
http://aserdtg.wikidot.com/,
http://dsgadtg.wikidot.com/,
http://eytder.wikidot.com/,
http://dsghsfh.wikidot.com/,
http://dsrww.wikidot.com/,
http://dsfzgds.wikidot.com/,
http://sdtgdf.wikidot.com/,
http://dszfas.wikidot.com/,
http://dfgfxd.wikidot.com/,
http://dsfdse.wikidot.com/,
http://sdrse.wikidot.com/,
http://eawfxf.wikidot.com/,
http://dssdfg.wikidot.com/,
http://dsfwef.wikidot.com/,
http://yyhfc.wikidot.com/,
http://fdsgd.wikidot.com/,
http://scxdfgdf.wikidot.com/,
http://dfsdraa.wikidot.com/,
http://fghdftgsd.wikidot.com/,
http://fsfplk.wikidot.com/,
http://gzfolk.wikidot.com/,
http://stfplk.wikidot.com/,
http://szfplk.wikidot.com/,
http://zhfplk.wikidot.com/,
http://zqfplk.wikidot.com/,
http://zjfplk.wikidot.com/,
http://sgfplk.wikidot.com/,
http://mmfplk.wikidot.com/,
http://jmfplk.wikidot.com/,
http://hyfplk.wikidot.com/,
http://hzfplk.wikidot.com/,
http://swfplk.wikidot.com/,
http://mzfplk.wikidot.com/,
http://yjfplk.wikidot.com/,
http://zsfplk.wikidot.com/,
http://nnfplk.wikidot.com/,
http://qyfplk.wikidot.com/,
http://glfplk.wikidot.com/,
http://dgfplk.wikidot.com/,
http://bhfplk.wikidot.com/,
http://lzfplk.wikidot.com/,
http://qzfplk.wikidot.com/,
http://rlfplk.wikidot.com/,
http://wzfplk.wikidot.com/,
http://ncfplk.wikidot.com/,
http://hezfplk.wikidot.com/,
http://bsfplk.wikidot.com/,
http://fcgfplk.wikidot.com/,
http://ggfplk.wikidot.com/,
http://ganzfolk.wikidot.com/,
http://ycfplk.wikidot.com/,
http://xxfplk.wikidot.com/,
http://pxfplk.wikidot.com/,
http://jafplk.wikidot.com/,
http://srfplk.wikidot.com/,
http://jjfplk.wikidot.com/,
http://fzfplk.wikidot.com/,
http://fuzfplk.wikidot.com/,
http://jdzfplk.wikidot.com/,
http://zzfplk.wikidot.com/,
http://xmfplk.wikidot.com/,
http://quzfplk.wikidot.com/,
http://ptfplk.wikidot.com/,
http://lyfplk.wikidot.com/,
http://czfplk.wikidot.com/,
http://tyfplk.wikidot.com/,
http://shzfplk.wikidot.com/,
http://yqfplk.wikidot.com/,
http://dtfplk.wikidot.com/,
http://csfplk.wikidot.com/,
http://zjjfplk.wikidot.com/,
http://huyfplk.wikidot.com/,
http://xtfplk.wikidot.com/,
http://zzhfplk.wikidot.com/,
http://xyfplk.wikidot.com/,
http://ximfplk.wikidot.com/,
http://whfplk.wikidot.com/,
http://syfplk.wikidot.com/,
http://hsfplk.wikidot.com/,
http://hgfplk.wikidot.com/,
http://shzhfplk.wikidot.com/,
http://xgfplk.wikidot.com/,
http://xnfplk.wikidot.com/,
http://xzfplk.wikidot.com/,
http://zgfplk.wikidot.com/,
http://guyfplk.wikidot.com/,
http://luzfplk.wikidot.com/,
http://deyfplk.wikidot.com/,
http://cdfplk.wikidot.com/,
http://suzfplk.wikidot.com/,
http://qrfplk.wikidot.com/,
http://hffplk.wikidot.com/,
http://fgfplk.wikidot.com/,
http://fdgfplk.wikidot.com/,
http://aqfplk.wikidot.com/,
http://fhfplk.wikidot.com/,
http://masfplk.wikidot.com/,
http://xbfplk.wikidot.com/,
http://xcfplk.wikidot.com/,
http://yuntfplk.wikidot.com/,
http://weihfplk.wikidot.com/,
http://qdfplk.wikidot.com/,
http://jnfplk.wikidot.com/,
http://dyfplk.wikidot.com/,
http://lwfplk.wikidot.com/,
http://hezefplk.wikidot.com/,
http://wffplk.wikidot.com/,
http://zbfplk.wikidot.com/,
http://zgxfplk.wikidot.com/,
http://dezfplk.wikidot.com/,
http://lcfplk.wikidot.com/,
http://lffplk.wikidot.com/,
http://tafplk.wikidot.com/,
http://zhzhfplk.wikidot.com/,
http://zhengzfplk.wikidot.com/,
http://pdsfplk.wikidot.com/,
http://xinxfplk.wikidot.com/,
http://luoyfplk.wikidot.com/,
http://kffplk.wikidot.com/,
http://yyfplk.wikidot.com/,
http://xucfplk.wikidot.com/,
http://jzuofplk.wikidot.com/,
http://hbfplk.wikidot.com/,
http://anyfplk.wikidot.com/,
http://zkfplk.wikidot.com/,
http://xinyfplk.wikidot.com/,
http://sxmfplk.wikidot.com/,
http://sqfplk.wikidot.com/,
http://lhfplk.wikidot.com/,
http://thfplk.wikidot.com/,
http://spfplk.wikidot.com/,
http://liaoyfplk.wikidot.com/,
http://jlfplk.wikidot.com/,
http://ccfplk.wikidot.com/,
http://qqhefplk.wikidot.com/,
http://hegfplk.wikidot.com/,
http://jmsfplk.wikidot.com/,
http://hebfplk.wikidot.com/,
http://dqfplk.wikidot.com/,
http://yucfplk.wikidot.com/,
http://qthfplk.wikidot.com/,
http://mdfplk.wikidot.com/,
http://jxfplk.wikidot.com/,
http://hehfplk.wikidot.com/,
http://wxfplk.wikidot.com/,
http://taizfplk.wikidot.com/,
http://suzhfplk.wikidot.com/,
http://ntfplk.wikidot.com/,
http://njfplk.wikidot.com/,
http://hzhfplk.wikidot.com/,
http://lygfplk.wikidot.com/,
http://xuzfplk.wikidot.com/,
http://yancfplk.wikidot.com/,
http://yzhfplk.wikidot.com/,
http://wzhfplk.wikidot.com/,
http://jiaxfplk.wikidot.com/,
http://zxfplk.wikidot.com/,
http://nbfplk.wikidot.com/,
http://huzfplk.wikidot.com/,
http://hjfplk.wikidot.com/,
http://trfplk.wikidot.com/,
http://tzfplk.wikidot.com/,
http://weizfplk.wikidot.com/,
http://zhsfplk.wikidot.com/,
http://lpsfplk.wikidot.com/,
http://guizfplk.wikidot.com/,
http://bjfplk.wikidot.com/,
http://ansfplk.wikidot.com/,
http://zyfplk.wikidot.com/,
http://ztfplk.wikidot.com/,
http://rxfplk.wikidot.com/,
http://qjfplk.wikidot.com/,
http://ljfplk.wikidot.com/,
http://kmfplk.wikidot.com/,
http://xiyfplk.wikidot.com/,
http://wnfplk.wikidot.com/,
http://xafplk.wikidot.com/,
http://bjifplk.wikidot.com/,
http://tcfplk.wikidot.com/,
http://tsfplk.wikidot.com/,
http://lzhfplk.wikidot.com/,
http://jygfplk.wikidot.com/,
http://jcfplk.wikidot.com/,
http://byfplk.wikidot.com/,
http://zhyfplk.wikidot.com/,
http://plfplk.wikidot.com/,
http://wwfplk.wikidot.com/,
http://jqfplk.wikidot.com/,
http://hmfplk.wikidot.com/,
http://shyfplk.wikidot.com/,
http://fusfplk.wikidot.com/,
http://dlfplk.wikidot.com/,
http://bxfplk.wikidot.com/,
http://asfplk.wikidot.com/,
http://liayfplk.wikidot.com/,
http://fxfplk.wikidot.com/,
http://ykfplk.wikidot.com/,
http://jzfplk.wikidot.com/,
http://ddfplk.wikidot.com/,
http://cyfplk.wikidot.com/,
http://hhhtfplk.wikidot.com/,
http://pjfplk.wikidot.com/,
http://hldfplk.wikidot.com/,
http://tlfplk.wikidot.com/,
http://wuhfplk.wikidot.com/,
http://tonglfplk.wikidot.com/,
http://tjfplk.wikidot.com/,
http://btfplk.wikidot.com/,
http://cffplk.wikidot.com/,
http://wlmqfplk.wikidot.com/,
http://sjzfplk.wikidot.com/,
http://lsfplk.wikidot.com/,
http://hdfplk.wikidot.com/,
http://chdfplk.wikidot.com/,
http://mxcsa.wikidot.com/,
http://wxzaz.wikidot.com/,
http://zxsa.wikidot.com/,
http://zsdc.wikidot.com/,
http://zxcvh.wikidot.com/,
http://sddgh.wikidot.com/,
http://zdsw.wikidot.com/,
http://dfsw.wikidot.com/,
http://dsgf1.wikidot.com/,
http://s44as.wikidot.com/,
http://gzxd.wikidot.com/,
http://fgse.wikidot.com/,
http://rdcs.wikidot.com/,
http://fsec.wikidot.com/,
http://mkhy.wikidot.com/,
http://nbvc.wikidot.com/,
http://cvfd.wikidot.com/,
http://bcvf.wikidot.com/,
http://fdty.wikidot.com/,
http://ufgh.wikidot.com/,
http://hifv.wikidot.com/,
http://ytre.wikidot.com/,
http://tyug.wikidot.com/,
http://oytf.wikidot.com/,
http://sdeo.wikidot.com/,
http://hjsa.wikidot.com/,
http://fdgd.wikidot.com/,
http://fgrw.wikidot.com/,
http://ghfo.wikidot.com/,
http://ngiw.wikidot.com/,
http://jybf.wikidot.com/,
http://fvup.wikidot.com/,
http://hjfy.wikidot.com/,
http://pdun.wikidot.com/,
http://cvbs.wikidot.com/,
http://fdgdfsgdf.wikidot.com/,
http://xzczs.wikidot.com/,
http://dcvxc.wikidot.com/,
http://sdd4ds.wikidot.com/,
http://jkyv.wikidot.com/,
http://fadsgwsea.wikidot.com/,
http://sadas.wikidot.com/,
http://ffdg1df.wikidot.com/,
http://migr.wikidot.com/,
http://ffdgdf.wikidot.com/,
http://czsa.wikidot.com/,
http://hqyv.wikidot.com/,
http://lfdsds.wikidot.com/,
http://fghfh.wikidot.com/,
http://hghfg.wikidot.com/,
http://fdstgf.wikidot.com/,
http://jgsw.wikidot.com/,
http://piyr.wikidot.com/,
http://ghfs.wikidot.com/,
http://fjtv.wikidot.com/,
http://taobao.wikidot.com/,
http://chengbang.ccedpw.com/,
http://nanzhuo3.shop.bxgtd.com/,
http://qiaojiao778.shop.bxgtd.com/,
http://doushi4566.shop.bxgtd.com/,
http://taodao.shop.bxgtd.com/,
http://jidun84.shop.bxgtd.com/,
http://tianquang67.shop.bxgtd.com/,
http://bojuenu1639.shop.bxgtd.com/,
http://cengtied15.shop.bxgtd.com/,
http://yeniangzh49.shop.bxgtd.com/,
http://www.andu729.yhtx.tv/,
http://www.tangyan83.yhtx.tv/,
http://www.ditang2.yhtx.tv/,
http://www.kaonai43043.yhtx.tv/,
http://www.duishuo.yhtx.tv/,
http://www.pabenye4186.yhtx.tv/,
http://www.waxuanqi61.yhtx.tv/,
http://www.haoganson86.yhtx.tv/,
http://www.piaoqinc13.yhtx.tv/,
http://www.qianghuai78.yhtx.tv/,
http://www.shenxiao34.yhtx.tv/,
http://www.fengniuge59.yhtx.tv/,
http://www.daojiongr71.yhtx.tv/,
http://www.sunkaizu34.yhtx.tv/,
http://www.cengshes84.yhtx.tv/,
http://www.51xsq.com/qy/yunqiang208/,
http://www.51xsq.com/qy/youshi66/,
http://www.51xsq.com/qy/kongwei55447/,
http://www.51xsq.com/qy/wendou/,
http://www.51xsq.com/qy/bibei8414689/,
http://www.51xsq.com/qy/hudiecu7642/,
http://www.51xsq.com/qy/tuncangt95/,
http://www.51xsq.com/qy/jianniuda12/,
http://www.51xsq.com/qy/yanpaijio77/,
http://www.51xsq.com/qy/feixiong68/,
http://www.51xsq.com/qy/tuofengj14/,
http://www.51xsq.com/qy/mengsong30/,

0