Blogs

记录一下自己涉及的一些的科研方向

高性能网包处理

网包分类(Packet Classification)

这是网络设备的最最基本的要求! 狭义理解可以把一般性的网络转发:最长前缀匹配(LPM), 看作其子类。

• 输入: 输入5元组(ip,port,protocol)规则集, 或者是OpenFlow这种自定义协议
• 输出: 匹配的规则(可以是最高优先级、也可以是匹配的所有规则)
• 目标: 低时延、高吞吐(可能通过牺牲时延得到提升, 如batch)、更新响应快
• 理论: 计算几何和数据库查询
• 实验室背景: 实验室的老方向,进行了10年左右的research。
• 方案: TSS、HyperSplit、HiCuts等算法，为了高性能、一般使用树、哈希作为典型数据结构，利用分组、启发式进行决策树分割等方法进行优化。在硬件方向，一般会使用FPGA、TCAM进行硬件优化。
• 前沿: 优化或者使用混合方法。优化的典型是NeuroCuts，使用RL来优化决策树的构造; 混合方法的典型是CutSplit、TSS和树混合等。

正则匹配(Regular Expression)

在网络检测中基本的功能

• 输入: 正则表达式规则集、字符串(流式)
• 输出: 命中的规则
• 目标: 低时延、高吞吐、更新(更新一般是个次要因素)
• 理论: 自动机理论、BDD优化等
• 实验室背景: 实验室之前的博士生研究了7年左右的方向。
• 方案: DFA和NFA是最基本的两种方法, 基于这两种方法衍生出来了Hybrid-FA、XFA(增加跳转优化)等, 核心解决DFA的内存爆炸和NFA的匹配过慢的问题, 其中还涉及到分组算法以及利用字符串匹配来做预过滤的工程优化。
• 前沿: Intel的Hyperscan库

网包调度(Packet Scheduling)

• 输入: 各种流量、人为定义的性能要求
• 输出: 按照一定顺序调度流量
• 理论: 微分方程(离散网包控制进行连续化–使用窗口)和各种调度理论(未来有可能使用ML优化，但其高性能的要求使得未必工作OK–需要考虑应用背景)。
• 实验室背景: 自己从大四开始陆陆续续做了大约1年的方向
• 方案: 典型的令牌桶(Token Bucket)、虚拟时钟(Virutal Clock)，加上层级架构和典型的服务调度算法，从而诞生的HTB、WFQ等算法。在硬件方向，使用FPGA做优化(已经有2篇网络顶会Sigcomm/NSDI的文章了)。
• 前沿: PIFO引发的统一性的算法设计，这里存在问题的是TB能应对抖动保持高性能，但PIFO(或者说背后的Virtual Clock)应对burst存疑。

网络管理功能

网络功能验证(Network Verification)

• 实验室背景: 跟随同事参与了解一些东西。

涵盖的内容有：数据平面的验证(针对网络拓扑和netflow)、控制平面的验证(针对网络功能,例如NFV和Protocol)。典型的算法有：HSA、Batfish等。现在最新的文章聚焦在使用SAT等形式化验证的PL常见内容来进行
网络建模(Network Modelling)可以认为是网络验证的第一步

我非常喜欢的网络小说作家“愤怒的香蕉”，喜欢在每年生日的时候写一下总结和随笔；我觉得这是一个很好的习惯，所以从今年开始我也尝试每年在生日的时候总结过去、展望未来，写写自己的想法和感悟。

日月如梭

我现在是真真正正体会到什么叫做日月如梭，光阴似箭了。我竟然一眨眼已经到了22岁，研究生也已经迈入2年级，我忽然间觉得自己老了许多，总有种步入中年的感觉。

大学时期的我做事情基本不怎么着急，觉得自己有大把的时间，世界有很多值得探索的地方；但最近一段时间，自己越来越感觉到自己的一事无成、越来越焦躁和功利性、越来越难以专注地长时间学习。

但这一点我觉得也未必是坏事，毕竟“从长期看，我们都死了”。很多人会讲一些大道理，强调所谓的“后天很美好”，但是现实世界往往是“后年很美好”，乃至“10年后很美好”、“死后很美好”。我们把眼界放到整个社会，很多事情是有价值的、需要有人做的；但如果我们把眼界放到个人，可能所谓“实用性”、“功利性”更加重要。如何在个人发展中平衡“圣人”的“人性”、道德和“常人”的“兽性”、贪婪，是非常重要的事情。毕竟自己现在是入世之人，阳春白雪和饮食男女都是需要一定关注的事情。

缺乏坚持

在过去的1年间，对于那些长时间投入的事情，自己半途而废的情形频繁出现：英语、健身、良好作息和良好时间管理。持续性混吃等死、间歇性踌躇满志，这句话对我来说真是扎心。

健身断断续续进行了半年，在腹肌初现的时候又暂停了1个月，体重迅速增加，逼近130斤；英语学习断断续续进行了2个月，英语口音还是没有改过来。我可以骄傲地说，贡献了健身房和流利说的盈利。

但自己也还是有收获的，健身找到了门路也起到了一定效果，英语阅读能力得到很好提升。后面自己要做的第1件事就是把这些事情的坚持常态化，坚持每天锻炼1小时、阅读1小时、课外学习1小时、睡眠9小时、工作8小时的节奏，坚持每日、每周和每月的总结。

三观的塑造

自己的三观在这1年间渐渐走向定型，感谢我在过去的1年间了解一些康德的思想和“政事堂”的坚持追更。现在的我在世界观上推崇康德，对于所谓的政治、管理有了些认知。未来自己需要在这方面继续努力。

最近, 有三则新闻都非常火爆:

1. 华为胡玲发文
2. 清华学生的《浅谈清华学风、课程内卷、特奖》
3. 山大的留学生又引起波澜

管理是门艺术

当很多人聚集在一起后，就产生管理、政治。我现在越发感觉针对于人的“管理科学”与“政治科学”远比“自然科学”，对个人的发展来说，更加重要；当然，后者对于整个社会的扩大很重要。前者分蛋糕，后者做大蛋糕；分蛋糕的人管理着做蛋糕的人。对于一个时代的人来说，“不患寡而患不均”；而要实现整体的“跨越式发展”，有时候“让一部分富起来，先富带动后富”就更加重要。但是，很少有人能看清楚这一点，或者超越自己的屁股，无论是“先富”还是“先贫”。

管理，有两个方向：对内和对外；有两个维度：对上和对下。怎么平衡这些关系，是非常复杂的问题；更何况还要考虑到人的不理性的问题。

大人, 时代变了

90后、00后正在崛起，这一代人明显与上一代人有较大的成长背景差异: 衣食无忧、崇尚独立、更加自信，这些人进入到社会以及成为社会主流后，会产生明显的不同。

Patience, the ability to put your desires on hold for a time, is a presious and rare virtue.

Cool things

Consumer Electronics

Personal Desire

• 男士护肤: 养成用大宝、护手霜的习惯
• 男士香水: 知乎上有各种推荐, 用起来感觉不错～

随身物品

1. 文具类

• 笔(红、黑各一支+备用)
• 本子(日常使用的记录本、草稿纸)
• 瑞士军刀
• 安卓平板(划掉)

2. 生活类

• 垃圾袋
• 口香糖
• 雨伞
• 水杯
• 胃药
• 钱包(证件)
• 手套(冬)
• 护手霜
• 纸巾、湿巾

Background

It is well known that the GFW makes it difficult to use Google which is necessary for research. With National Day coming, many IPs are blocked. However, the IPv6 address is not blocked, perhaps because the address is too long to be matched fast.

Anyway, we can not connect to the VPS via IPv4, but via IPv6. It is reasonable for us to buy a VPS with IPv6 address and reach Google Scholar with it.

Buy VPS

There are many merchants, providing VPS. You can find these merchants in many websites, such as VPS Recommend and Zhujiceping. What’s more, you can get similar information by searching in Github. I use the Ramnode for 2 years and it works well.

How to choose one

If we ignore the overselling problem, which is the main reason for bad performance under normal conditions, the main features we need to consider about are as follows.

1. Latency. It is most important!!! However, because of the physical links and government regulation, the latency from mainland of China to VPS in Japan or Singapore is more than 100ms. HongKong, China is better, but still needs 50ms or more. As a result, if you want to play online games, VPS is not suitable for you but some other technologies.
2. CPU and RAM. If you just install V2ray and other simple softwares, 128M RAM is enough. If you want your own blog on the VPS, better CPU and more RAM is necessary.
3. Bandwidth. It is least important. Because even 500GB is enough for private use but if you would like to see many academic videos on Youtube, you need more bandwidth as 20-minutes HD video needs 1GB or more.

To better understand the difference of VPS, some virtualization technologies are vital. There are three main virtualization technologies used in VPS: Xen, KVM and OpenVZ. You can get more information from this link.

1. OpenVZ, in fact, is not virtualization technology but Linux-based containerization technology. You can not run Docker on it or modify kernel modules, but OpenVZ VPS is cheapest.
2. Xen is an open-source virtualization platform that originally began as a research project in the University of Cambridge. It uses QEMU to provide complete hardware emulation.
3. KVM is a kernel module, and reuses many functions of the Linux kernel for its purposes. It only runs on hardware that supports hardware-assisted virtualization, and is common.

Buy Domain Name

It is necessary to buy a domain name because it is hard for us to remember 120.12.6.78 or 2340:234:be::12f. Besides, we can just change the DNS configuration without changing the configuration of our programs, which is very convenient. You can get a ‘.site’ domain for 10 year with about $30, which is not very expensive.

Install V2ray

You can install v2ray buy run the go.sh from https://install.direct/go.sh and refer to the official doc.

After installation, you need to configure the config.json. I recommend to use the mKCP or QUIC,which is faster, for the connection between client and VPS. You can get the configuration from Internet easily. The inbound configuration in VPS and the outbound configuration must be the same, for example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

"outbounds": [{
   "protocol": "vmess",
   "settings": {
       "vnext": [
           {
               "address": ["123::00"],
               "port": [123],
               "users":[
                   {
                       "id": ["4684-4565"],
                       "level": 1,
                       "alterId": 64
                   }
               ]
           }
       ]},
       "streamSettings": {
       "network": "mkcp",
       "kcpSettings": {
         "uplinkCapacity": 5,
         "downlinkCapacity": 100,
         "congestion": true,
         "header": {
           "type": "none"
         }
       }
     }
 }]

The configuration of inbound in client can be personalized with socks or http, with authentication or without, for example:

1
2
3
4
5
6
7
8
9
10
11

"inbounds": [{
   "port": [123],
   "protocol": "socks",
   "sniffing":{
       "enabled": true,
       "destOverride": ["http", "tls"]
   },
   "settings": {
       "auth": "noauth"
   }
 }],

The configuration of outbound in VPS needs to be as follows:

1
2
3
4
5
6

"outbounds": [
    {
      "protocol": "freedom",
      "settings": {}
    }
  ]

Configure IPv6 in Aliyun

IPv6 is supported by China Mobile and other Mobile Network Operators. However, we can not use IPv6 sometimes. So we may need another server in China:

Client <– IPv4 –> China Server <– IPv6 –> Abroad Server

The ‘China Server’ need to support both IPv4 and IPv6, but it is hard in China now. Aliyun, Tencent Cloud and some other Cloud provider do not support IPv6 directly. So we need to use tunnel: 6in4. We can use the tunnel broker of HE.

Enable IPv6 in Aliyun

Modify the /etc/sysctl.conf to support IPv6:

1
2
3

net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0

Restart network to support IPv6, and check with ifconfig.

Register Tunnel Broker

The website is https://www.tunnelbroker.net/ and make a registeration. Create a regular tunnel and use the public IP of Aliyun server to generate a tunnel. After this, you can get an example configurations for Linux-route2, which is suitable for CentOS. You need to change the public IP into private IP in Aliyun.

1
2
3
4
5
6

modprobe ipv6
ip tunnel add he-ipv6 mode sit remote 66.220.18.42 local [Private IP] ttl 255
ip link set he-ipv6 up
ip addr add 2001:470:c:17cb::2/64 dev he-ipv6
ip route add ::/0 dev he-ipv6
ip -f inet6 addr

Use ping6 command to check and try to ping your VPS abroad. And install V2ray in Aliyun too and make the inbound configuration as http or socks. You can use vmess or mKCP too if you would like. Finally, you get the following infrasturature and are able to reach Google Scholar.

Client <– Socks –> China Server <– V2ray –> Abroad Server

学习DPDK的相关知识，使用《深入浅出DPDK》这一本书和DPDK官网的document足够了。

安装

1. 从DPDK官网下载LTS版本
2. 安装依赖(主要是gcc\numa-dev\kernel-dev)
3. 解压DPDK源码后进入tool文件夹下，执行对应安装的sh脚本，根据指示编译源码、安装驱动、绑定PCI网卡设备、配置大页
4. export RTE_SDK和RTE_TARGET
5. 根据examples文件夹下的程序对应修改即可

代码使用注意事项

1. 要理解mempool的概念(大页内存的使用)和rte_mbuf的申请、回收
2. 要理解调用EAL做argv解析和examples给出的网卡初始化过程中的一系列事宜: 分配Queue给Rx、分配TxBuffer，设置端口性质等。可以参考博文
3. 定义结构体的时候, 要通过attribute((packed)) 来说明结构体不进行Cache对齐，否则gcc优化会自动进行CacheLine对齐为4B整数倍。相关说明参考博文

常见命令

文件位置说明

Grub引导

Grub是一个linux引导程序

待办：grub的实际运行原理调研

EFI,GPT问题

问题解决参照 www.mintos.org/skill/rescue-efi-grub.html

1
2
3
4
5
6
7
8
9

mount /dev/sda4 /mnt（注意先确认自己的 / 分区是 sdaX）  
mount /dev/sda1 /mnt/boot/efi  
mount -t proc proc /mnt/proc  
mount -t sysfs sys /mnt/sys  
mount -o bind /dev /mnt/dev  
mount -t devpts pts /mnt/dev/pts/  
chroot /mnt  
grub-install /dev/sda1  
update-grub2

或者，先添加ubuntu启动项到efi中，再修复

1
2
3

mount /dev/sdb4 /mnt/system  
mount /dev/sdb1 /mnt/system/boot/efi  
efibootmgr -c -d /dev/sdb -p 2 -w -L ubuntu

重启进入ubuntu后，grub-install /dev/sda1； update-grub2

Vim使用

Command

[num]+command:重复某个命令num次

[start postion]+command+[end postion]:
dt”:删除所有字符直到”

块操作

块操作，典型的操作： 0 C-v C-d I– [ESC]
^ → 到行头
C-v → 开始块操作
C-d → 向下移动 (你也可以使用hjkl来移动光标，或是使用%，或是别的)
I– [ESC] → I是插入，插入“–”，按ESC键来为每一行生效。

Setting files

分屏

Reference

1. https://coolshell.cn/articles/5426.html

Welcome to My Blog. This is my very first post. I hope that I can help someone.

Blog Initailization

My Blog is based on the Hexo, thanks to the Github Pages.
To beautify my blog, I use the theme: next and modify some settings.
We need to use git clone https://github.com/theme-next/hexo-theme-next themes/next to install.
After the Installation, I modify the Personal Information and other configurations according to Instructions.

Personalization

I add ‘Blog Life’, ‘Visitor number’ and ‘Personal Information’.
The key is to modify the _config.yml of Next and Hexo.
And with the help of js codes, we can add some interesting functions.

Menu

1. Create the files
   Files are stored in ./source/: hexo new page “about”.
   We can create a markdown file in ./source/about and modify it. Btw, blog articles are in ./source/_posts.
   I add my CV in the about menu.

2. Remember to modify the Next config of ‘menu’.

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

   menu: home: / || home about: /about/ || user tags: /tags/ || tags categories: /categories/ || th archives: /archives/ || archive #schedule: /schedule/ || calendar #sitemap: /sitemap.xml || sitemap #commonweal: /404/ || heartbeat avatar: # in theme directory(source/images): /images/avatar.gif # in site directory(source/uploads): /uploads/avatar.gif # You can also use other linking images. - url: /images/avatar.gif + url: /images/header.jpg # Picture link # If true, the avatar would be dispalyed in circle. rounded: false # The value of opacity should be choose from 0 to 1 to set the opacity of the avatar. opacity: 1 # If true, the avatar would be rotated with the cursor. rotated: false

Search

1. To enable search function, run: npm install hexo-generator-searchdb –save
2. Modify Hexo config of ‘search’.

   1 2 3 4 5

   search: # add the search function path: search.xml field: post format: html limit: 10000

RSS

1. Install the plugin, run: npm install –save hexo-generator-feed
2. Modify the Hexo and Next config.

   1 2 3 4 5

   # In Hexo plugins: hexo-generate-feed # In Next rss: /atom.xml

Count

1. Install the plugin, run: npm install hexo-symbols-count-time –save
2. Modify the Hexo config and Next.

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   # In Hexo symbols_count_time: symbols: true time: true total_symbols: true total_time: true # In Next busuanzi_count: enable: true total_visitors: true total_visitors_icon: user total_views: true total_views_icon: eye post_views: true post_views_icon: eye

Run time

1. Modify /blog/themes/next/layout/_partials/footer.swig, add the codes:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

   <div id="days"></div> <script> function show_date_time(){ window.setTimeout("show_date_time()", 1000); BirthDay=new Date("04/17/2018 15:13:14");//修改为自己的blog建站时间 today=new Date(); timeold=(today.getTime()-BirthDay.getTime()); sectimeold=timeold/1000 secondsold=Math.floor(sectimeold); msPerDay=24*60*60*1000 e_daysold=timeold/msPerDay daysold=Math.floor(e_daysold); e_hrsold=(e_daysold-daysold)*24; hrsold=setzero(Math.floor(e_hrsold)); e_minsold=(e_hrsold-hrsold)*60; minsold=setzero(Math.floor((e_hrsold-hrsold)*60)); seconds=setzero(Math.floor((e_minsold-minsold)*60)); document.getElementById('days').innerHTML="本站已安全运行"+daysold+"天"+hrsold+"小时"+minsold+"分"+seconds+"秒"; } function setzero(i){ if (i<10) {i="0" + i}; return i; } show_date_time(); </script>

2. In the markdown files, we can add the js codes. For example, with girlfriend.

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   <div id="withGirlFriend"></div> <script type="text/javascript"> var now = new Date(); function createtime() { var grt= new Date("10/18/2016 22:30:00"); now.setTime(now.getTime()+250); days = (now - grt ) / 1000 / 60 / 60 / 24; dnum = Math.floor(days); hours = (now - grt ) / 1000 / 60 / 60 - (24 * dnum); hnum = Math.floor(hours); if(String(hnum).length ==1 ){hnum = "0" + hnum;} minutes = (now - grt ) / 1000 /60 - (24 * 60 * dnum) - (60 * hnum); mnum = Math.floor(minutes); if(String(mnum).length ==1 ){mnum = "0" + mnum;} seconds = (now - grt ) / 1000 - (24 * 60 * 60 * dnum) - (60 * 60 * hnum) - (60 * mnum); snum = Math.round(seconds); if(String(snum).length ==1 ){snum = "0" + snum;} document.getElementById("withGirlFriend").innerHTML = "和女友在一起"+dnum+" 天 " + hnum + " 小时 " + mnum + " 分 " + snum + " 秒"; } setInterval("createtime()",250); </script>