Only for learning and exchange, non-profit, intrusion and deletion (dog head saves life)

I. Overview

1.1 Effect

In general, this method is accomplished through image recognition , without intruding into the game, without reading memory, and without being detected safely.

1.2 Prerequisite knowledge

1. There are a variety of different firearms in the game. Different firearms have different recoil and different firing rates. The same firearm, after installing different accessories, the recoil will also change.
2. The upward movement of the y-axis of the firearm is fixed, and the x-axis is random, so our program only moves the y-axis of the mouse. Manual operation in the x-axis game.

1.3 Brief description of the realization principle

1. Monitor the keyboard and mouse through the pynput module in python.

Listen for the left mouse button to be pressed, and start moving the mouse at this time. Lift up the left button to stop moving.
Monitor the keyboard keys, such as the tab key, open the backpack at this time , take a screenshot and start to identify the equipment column.

2. Use python's pyautogui module to take screenshots, and you can capture the screen at a specified location.

3. The intercepted pictures are processed through python's opencv module.

4. Use the SSIM algorithm to compare the similarity of the pictures, and obtain the weapons and accessories in the equipment bar.

5. Manipulate mouse movement through python's pydirectinput.

Two, detailed explanation

2.1 pynput monitor keyboard

import pynput.keyboard as keyboard

# 监听键盘
def listen_keybord():
    listener = keyboard.Listener(on_press=onPressed, on_release=onRelease)
    listener.start()

The listener of pynput is an asynchronous event, but it will be blocked, so if the event processing event is too long, it must be processed asynchronously.

2.2 Listening to events

The c_equipment class was created to encapsulate weapon information.
The focus is on the monitoring of the tab key, using asynchronous to detect equipment information.

def onRelease(key):
    try:
        if '1' == key.char:
            c_equipment.switch = 1 #主武器1
        elif '2' == key.char:
            c_equipment.switch = 2 #主武器2
        elif '3' == key.char:
            c_equipment.switch = 3 #手枪 switch=3的时候不压枪
        elif '4' == key.char:
            c_equipment.switch = 3 #刀具
        elif '5' == key.char:
            c_equipment.switch = 3 #手雷
    except AttributeError:
        if 'tab' == key.name:      #tab键异步操作检测
            asyncHandle()
        elif 'num_lock' == key.name:  #小键盘锁用来控制程序开关
            changeOpen()
        elif 'shift' == key.name:   
            c_contants.hold = False

2.3 pyautogui screenshot

To detect equipment, you must first take a screenshot when opening the equipment bar.

pyautogui.screenshot(region=[x, y, w, h])

x,y represent coordinates respectively, w,h represent width and height.
After intercepting, in order to compare pictures conveniently, the picture needs to be binarized and then saved locally.

The complete code is as follows:

import pyautogui

def adaptive_binarization(img):
    #自适应二值化
    maxval = 255
    blockSize = 3
    C = 5
    img2 = cv2.adaptiveThreshold(img, maxval, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, blockSize, C)
    return img2

# 屏幕截图
def shotCut(x, y, w, h):
    im = pyautogui.screenshot(region=[x, y, w, h])
    screen = cv2.cvtColor(numpy.asarray(im), cv2.COLOR_BGR2GRAY)
    temp = adaptive_binarization(screen)
    return temp
    
def saveScreen():
    screen1 = shotCut(1780, 125, 614, 570)
    cv2.imwrite("./resource/shotcut/screen.bmp", screen1)

2.4 Material preparation

After the screen shot is processed as above, before equipment recognition, we need to prepare a lot of material pictures for comparison.
For example: weapon name, buttstock, grip, muzzle

Weapon name:

gun stock

2.5 Crop Image

In order to facilitate picture comparison, we need to crop the captured picture of the equipment column into a picture of the same size as the material.

For example, we want to check the name of weapon one:

#读取之前的截屏
screen = cv2.imread("./resource/shotcut/screen.bmp", 0)
#裁剪出武器1名字
screenWepon1 = screen[0:40, 45:125]
#拿裁剪的图片和武器素材的目录作为入参，进行对比
w1Name = compareAndGetName(screenWepon1, "./resource/guns/")

2.6 Comparison pictures

#对比图片获取名字
def compareAndGetName(screenImg, dir):
    #获取目录下所有文件
    content = os.listdir(dir)
    name = 'none'
    max = 0
    #遍历文件
    for fileName in content:
        #使用opencv读取文件
        curWepone = cv2.imread(dir + fileName, 0)
        #使用SSIM算法拿到图片相似度
        res = calculate_ssim(numpy.asarray(screenImg), numpy.asarray(curWepone))
        #获取相似度最大的
        if max < res and res > 0.5:
            max = res
            name = str(fileName)[:-4]
    return name

SSIM algorithm:

def calculate_ssim(img1, img2):
    if not img1.shape == img2.shape:
        raise ValueError('Input images must have the same dimensions.')
    if img1.ndim == 2:
        return ssim(img1, img2)
    elif img1.ndim == 3:
        if img1.shape[2] == 3:
            ssims = []
            for i in range(3):
                ssims.append(ssim(img1, img2))
            return numpy.array(ssims).mean()
        elif img1.shape[2] == 1:
            return ssim(numpy.squeeze(img1), numpy.squeeze(img2))
    else:
        raise ValueError('Wrong input image dimensions.')

At this point, we can get the name of the weapon at position 1 of the equipment bar.

2.7 Operating the mouse

After knowing the name of the weapon, in the same way, we can obtain the accessories of the equipment.
Then, listen for left mouse button presses, and start moving the mouse down.

Let's take the m762 weapon as an example:

Rate of fire: 86, each bullet interval 86 milliseconds

Recoil:
[42, 36, 36, 36, 42, 43, 42, 43, 54, 55, 54, 55, 54, 55, 54, 55, 62, 62, 62, 62, 62, 62, 62, 62 ,62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 77, 78, 77, 78]

Indicates the distance that needs to be moved down the y-axis after each bullet is fired to counteract the recoil.

def moveMouse(): 
    #从识别的数据中，再更具当前选择的武器，获取此刻的武器（比如按下1键，武器装备栏1为m762，那么此时武器就是m762）
    curWepone = getCurrentWepone()
    if (curWepone.name == 'none'):
        return
    #基础y轴补偿（没任何配件）
    basic = curWepone.basic
    #射速
    speed = curWepone.speed
    startTime = round(time.perf_counter(), 3) * 1000
    for i in range(curWepone.maxBullets):
        #是否可以开火，比如左键抬起，就中断。
        if not canFire():
            break
        #系数，比如按住shift屏息，就需要再原来基础上乘1.33
        holdK = 1.0
        if c_contants.hold:
            holdK = curWepone.hold
        #乘以系数后实际的移动距离
        moveSum = int(round(basic[i] * curWepone.k * holdK, 2))
        while True:
            if (moveSum > 10):
                #移动鼠标
                pydirectinput.move(xOffset=0, yOffset=10, relative=True)
                moveSum -= 10
            elif (moveSum > 0):
                pydirectinput.move(xOffset=0, yOffset=moveSum, relative=True)
                moveSum = 0
            elapsed = (round(time.perf_counter(), 3) * 1000 - startTime)
            if not canFire() or elapsed > (i + 1) * speed + 10:
                break
            time.sleep(0.01)

while loop in code:

In fact, after the first bullet is fired, we only need to move down a distance of 42, and then calculate the waiting time (0.086-the time to move the mouse), and then shoot the second bullet, and so on.

The function of the while loop is to prevent the screen from shaking too much. Because the distance of 42 is moved directly, the game shakes a lot, so we move the mouse multiple times in an interval of 86 milliseconds.

The sleep function in python is not accurate, so we have to time it ourselves to prevent missing the time interval of each bullet.
There is another advantage of being inaccurate, random, and it happens that you don't need to prevent detection by yourself.

3. The most troublesome part

The recoil of each gun is different, we need to go to the training ground of the game by ourselves, adjust the bullets one by one, and obtain accurate compensation data.

Four, finally

Upload the code to gitee, and communicate with those who are interested.
https://gitee.com/lookoutthebush/PUBG