7.5 KiB
Future plan
Here list the future plan for Alas, sorted from easy to hard.
Alas has a lot of handful public functions and class methods, from object detection to map logics, don't handcraft them again.
-
Test chapter 6 when using 1 fleet
-
Add logic to chapter 2, 4, 5, when using 2 fleets
Chapter 1 to 5 are written in logic of 1 fleet, and I added the logic of 2 fleets in chapter 3, try to imitate that.
-
Add more maps, from 7-3 to 13-4
9-2, 9-3, 9-4, 10-1, 10-2, already done. Thanks Cerzo, Lanser, whoamikyo
Read doc/development.md for details.
Some new logics may not included, read module/map/map.py
-
Add support for hard mapAlready doneAlas can use the map file from normal mode, no need to rewrite
Add a new option in module/config/argparser.py, under the "main chapter".
Add the arg name into module/config/dictionary.py
Add arg into config/template.ini
Load it in module/config/config.py
Pass that parameter to
ensure_campaign_ui
in module/campaign/campaign_ui.py line 74 -
Add support for dorm
Add new file module/reward/dorm.py
There should be options: DORM_REWARD_INTERVAL (1 hour, 2 hour, 3 hour, 4 hour), USE_GEM_FOOD (yes, no)
Should able to pick up love and coin in dorm, may hard to pick all of them, not a big deal.
Call OCR to detect food remain and total
Don't use template matching to detect which food is available, because the naval curry change from time to time. The unavailable food items have a white overlay, make good use of that.
-
Adaptive parameters for perspective detection
When alas get an perspective error, it stops. Catch the errors, auto adjust the parameters and retry.
Read perspective_en.md, see how alas do map detection, and you need to have some basic knowledge of perspective.
-
Restart when get stuck or lost connection
Don't simply use
handle_popup_confirm()
, because many module also use that.After the restart, retreat from map, because alas can't resume from a unfinished map.
Alas have highly wrapped API, you may need to touch some low-level functions to do that.
When you're done, test the speed, see if it effects on the performance.
-
Try ascreencapAlready doneI notice that ALauto use ascreencap, test and see if it's faster than uiautomator2.
If so, add that option in GUI, and set to default.
-
Update cnocr
If we use the latest
cnocr
, we can recognize English letters better, get rid ofmxnet==1.4.1
, have smaller model size.But I already train 2 models under
cnocr==1.0.0
, namedstage
anddigit
, test if they are compatible. -
Re-train ocr models
The existing ocr model named
stage
anddigit
are poorly trained, you need to make sure letters are full-filled the image.The fat letters which is frequently used game is
Impact
The slim letters use in level and oil is
AgencyFB-Regular
The letters in research which looks cool is
MStiffHeiHK-UltraBold
It's better to use only one model to recognize all.
-
Add support research
Do this after updating cnocr and retraining models.
Imitating commission.py, use ocr to get research name, classify them, then choose
-
Re-fit the relationship between map swipe and screen swipe
This function is
_map_swipe
in module/map/camera.py line 36. If you want to focus camera from D3 to E4, then it will do a swipe (-200, -140). A simple linear fit turn(1, 1)
to be(-200, -140)
.I fit this in 7-2, but i found it no working well in some large maps. Alas have to call
focus_to_grid_center
in line 89 to fix camera position. But sometimes that failed, alas update map data to the wrong grids, and cause a series of error untilensure_edge_insight
is called.I do know that the relationship between map swipe and screen swipe is complex, but linear would be enough, because we won't swipe too far.
How close is the camera on the map will also effect the relationship, you can use the
mid
of vertical lines the measure that. So closer camera means larger diff in vertical.mid, and further swipe, try to fit the the exact relationship. -
Combine perspective.py and homg_trans_beta brunch in ALauto
asd111333 shared his thoughts on map detection to me. His project is doing so good when there is only few empty grids on screen which is the weakness of Alas, and ALauto don't have a global map view, which is the strengthen of Alas.
perspective.py can generate the camera setting needed in his approach, so no longer need to hard-code camera settings for different maps.
Alas work in 720p and ALauto works in 1080p, so there is a 1.5x scaling.
def get_homo_data(self, tile_width=209/1.5, tile_height=209/1.5):
hori = self.horizontal[0].add(self.horizontal[-1])
vert = self.vertical[0].add(self.vertical[-1])
src_pts = hori.cross(vert).points
x = len(self.vertical) - 1
y = len(self.horizontal) - 1
dst_pts = np.array([[0, 0], [x * tile_width, 0], [0, y * tile_height], [x * tile_width, y * tile_height]], dtype=float)
dst_pts += src_pts[0]
return [src_pts * 1.5, dst_pts * 1.5]
I replaced the assets in 720p, now can directly generate the parameter for
cv2.warpPerspective
def get_perspective(self):
tile_width = 140
tile_height = 140
# Generate homo from src_pts and dst_pts
perspective = Perspective(image, config=self.config)
hori = perspective.horizontal[0].add(perspective.horizontal[-1])
vert = perspective.vertical[0].add(perspective.vertical[-1])
src_pts = hori.cross(vert).points
x = len(perspective.vertical) - 1
y = len(perspective.horizontal) - 1
dst_pts = np.array([[0, 0], [x * tile_width, 0], [0, y * tile_height], [x * tile_width, y * tile_height]],
dtype=float)
dst_pts += src_pts[0]
homo, _ = cv2.findHomography(src_pts, dst_pts)
# Re-generate to align image
area = self.config.DETECTING_AREA
area = np.array([[area[0], area[1]], [area[2], area[1]], [area[0], area[3]], [area[2], area[3]]])
matrix = homo.dot(np.pad(area, ((0, 0), (0, 1)), mode='constant', constant_values=1).T)
x, y = matrix[0] / matrix[2], matrix[1] / matrix[2]
x, y = x - np.min(x), y - np.min(y)
homo, _ = cv2.findHomography(area, np.array([x, y]).T)
# Perspective transform
size = (np.ceil(np.max(x)).astype(int), np.ceil(np.max(y)).astype(int))
screen_trans = cv2.warpPerspective(np.array(perspective.image), homo, size)
screen_edge = cv2.Canny(screen_trans, 50, 100)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
screen_edge = cv2.morphologyEx(screen_edge, cv2.MORPH_CLOSE, kernel)
# Find an empty grid
res = cv2.matchTemplate(screen_edge, free_tile_center, cv2.TM_CCOEFF_NORMED)
_, similarity, _, location = cv2.minMaxLoc(res)
logger.info(f' free_tile_center_sim: {str(round(similarity, 3)).ljust(5, "0")}')
I imagined a way to use two approach. After entering map, call perspective.py to generate camera data, and use homo_trans till the end. If get any error, fallback to perspective.
This will be faster and with lower error rate, and get rid of the parameters of
scipy.signal.find_peaks
I got a problem now. Alas use the edge of the map to locate camera. If edge is not insight, locate camera from history swipe which may be unstable. When edges appear again, use that to correct camera location. So even if i use homo_trans, i still need to call perspective from time to time.