chenzou - Tumblr blog

[chenzou]

UX Design Research, Final

This is a whole space based object recognition and service design, you can get the electronic components or other tools you need anytime and anywhere and support to send them back to their original place, and support more in-depth detailed data monitoring. Part 1: Feature matching: The feature descriptor is mainly for the key point after the calculation, with a set of vectors to describe this key point, this description includes not only the key point, but also the key point around the pixel points that contribute to it. This description includes not only the key point but also the pixels around the key point that contribute to it. It is used as a basis for target matching, and also allows the key point to have more invariant characteristics, such as illumination changes, 3D viewpoint changes, etc. Match each feature point descriptor in a set of feature points with the closest feature point descriptor in another set, then rank the obtained distances, and finally select the feature with the shortest distance as the match between the two.

Part 2: Gesture recognition This step I envision is similar to shortcut key operation, matching common devices with basic gestures, and then commanding the robot movement through gestures.

Part 3: Motion principle The six-legged robot is used to move and grasp the required electronics, because of the city's six tentacles, its first two hands can be lifted and grasp objects at will, and can be controlled using voice recognition or joystick.

Part 4: Garbage classification I envision the principle of image recognition using garbage sorting. First, we collect the live screen data posted by the camera nodes and convert it to a format supported by the image input. Then, the images are input to the network for scaling, transposing, array expansion and other operations to obtain processing results. Then, the obtained object card image coordinates are converted back to the original image coordinates, and they are classified according to the set rules based on the recognized object names, and then the corresponding object category names are obtained. Then, they are framed out to show the names and parameters of the electronic originals to start the hexapod robot and input the corresponding commands. After the play is opened, place the target card in front of the camera, when the camera recognizes the garbage card, it will give the corresponding name in the screen, different categories are framed with different color rectangles, here you can add gestures to recognize or colors, invalidate the item for red, put it back to the original position for blue and other instructions.

Part 5: Extensions and Further The more advanced depth camera is mainly explored here, which is a complementary solution I envision because of his function we can quickly get detailed information about the whole space, such as color and location, and even get detailed point cloud images, if it can be connected with the crawling robot, it can supplement some functions, such as the previously lost equipment or small accessories hidden in the crevices, by the space scan can be found quickly.

[chenzou]

UX Design Research Part 5

This week I returned as much energy as possible to the user, because I was always clear about the prototype of my final presentation, and I developed some problems closely related to the product, such as classifying the kinds of electronic components that people most need to be identified, and my potential customers helped me a lot, such as problems I had not considered before, like the difficulty of achieving a satisfactory success rate in the case of identifying resistors, because most of them are extremely similar in shape and color, which requires strong algorithms and is indeed a challenge.

On the other hand, for some of the more popular issues, that is, non-absolutely core user issues, the development of a series of questions, such as whether as a college student is willing to use this interaction mode to learn electronic components, such as such issues a good questionnaire software is a good helper, but you need to pay a little fee, I probably got about 200 people responded, hoping to try this new interactive college student groups accounted for about seventy percent, the reasons for its lack of support mostly come from the inability to think about the actual results, I think it is best to use the initial product can then conduct a second wave of surveys

[chenzou]

UX Design Research,Part 3

Current detection systems can be roughly divided into two types: deep learning-based approaches; and heuristic feature-based approaches. Heuristic feature-based solutions generally represent candidate regions through empirically human-designed features, while deep learning-based approaches extract object features through multilayer neural networks to achieve multiple combinations of local features to classify objects. Deep learning features can be learned adaptively according to the data distribution, so deep learning techniques have gained some advantages over heuristic feature techniques in terms of detection accuracy in recent years.

I found a super-pixel feature extraction process for scene understanding tasks based on the NYU Depth V2 database, which implements labeling the categories of individual objects in a scene. The database can be used as a benchmark for validating scene understanding algorithms in complex indoor environments. 1449 color and depth images with 480×640 resolution are provided in the database along with manually labeled images of various objects in the scene.

Extracting data is a headache, involving many formulas and algorithms, and is the next thing I need to focus on.

y=f(b+x1w1+x2e2+...x9w9)

I have a robot-based camera, and a set of depth cameras, and I will be using one of these systems to complete the final project.

[chenzou]

UX Design part 3

During this week I have been thinking about the presentation of the final project and I wanted to harvest some interactive devices, so I learned about some basic projects based on Arduino and Raspberry Pi. I seek to try t expand the interesting applications.

I think for anyone who is new to Arduino or Raspberry Pi, there are a large number of electronic components that need to be touched and understood, such as various types of resistors, capacitors, different interfaces, LEDs and some other accessories. We can use deep learning neural networks with object recognition to try to identify electronic components and related possibilities, and can present basic parameters based on the recognition results, and may include detailed descriptions and usage analysis. This could serve the whole IDM, or perhaps as a device located somewhere for people to learn from.

The current algorithms for electronic component classification research are mainly divided into two categories, one is based on the traditional image classification method, firstly, input image, then extract the image features through a complex procedure, and then classify according to the extracted features. Another type of classification method is based on deep learning, where the input image is first input, then the features are automatically extracted by convolutional neural network to generate a feature map, and then the target is localized and classified according to the feature map.

[chenzou]

UX Design Research, Part 2

This week I focused on the idea with the interest idle viewers, mainly students, and I personally have the same realization, this is because I missed the previous Raspberry Pi basics class offered by Makerspace. But I realized that they tried to popularize these basic and important knowledge, you know, despite the presence of a large number of devices here, such as 3D printing and VR devices, etc., but for our major including myself, not everyone has a software and hardware background, you know IDM's course focuses on Arduino, it has the advantage of being too easy to get started, which is good for students with no background, but not good for improving development skills. But it is not conducive to improving development skills, the limited processing power of Arduino boards means they may not be able to handle more complex tasks, I don't think as a designer should be limited by this, I examined the more mainstream hardware and software systems in the market and tried to design some friendly introductory system for designers.

Above is my collection of related devices including Raspberry Pi and FPGA and their development languages, I tried to let the designers understand the system relationship quickly and to create something suitable for their requirements.

[chenzou]

UX Design Research 1

https://makerspace.engineering.nyu.edu/m

NYU Makerspace is a collaborative workspace located at the NYU Tandon School of Engineering. It provides students, faculty and staff with a variety of tools and resources to create, design and prototype their projects. The space includes 3D printers, laser cutters, electronic workbenches, sewing machines and woodworking tools.

In addition to the equipment, MakerSpace offers a series of workshops and events that teach users how to use the equipment and develop new skills. The space is open to all members of the NYU community, regardless of their level of expertise or field of study.

You know it's almost a treasure trove, a lot of devices, a lot of activity, and the fact that I'm having a hard time finding flaws in it, I'd like to start with deep learning to see what's desirable to help it, and for now I think using technology to do some user grooming is the necessary work, and of course it may be adjusted as we get deeper into the research.

So I took this opportunity to start understanding some algorithms related to extracting important information and images.

Before the emergence of CNN, image processing has always been a big problem, on the one hand, because the amount of data for image processing is too large, for example, a 512 x 512 grayscale image has 252144 input parameters, not to mention color images such as 1024x1024x3, which also leads to its processing cost is very expensive and inefficient. On the other hand, it is difficult to guarantee the original features of the image during the digitization process, which also leads to a low accuracy of image processing.

Image classification: CNNs can be trained to classify images into various categories such as identifying objects, recognizing faces, detecting diseases in medical images, and identifying characters in handwriting.

Object detection: CNNs can help detect the presence and location of objects in an image or video. This is useful for applications such as self-driving cars, surveillance systems, and robotics.

Image segmentation: CNNs can help segment an image into different regions or objects, making it easier to identify and analyze specific parts of an image.

Style transfer: CNNs can be used to transfer the style of one image onto another, creating new and interesting visual effects.

Video analysis: CNNs can help analyze videos by detecting and tracking objects, recognizing actions and gestures, and predicting future frames.

[chenzou]

Conpsule 3

More than a hundred years have passed, the Earth's climate has changed dramatically, huge aliens with monsters invaded the Earth, ordinary residents were wantonly bullying the Earth's only remaining heroes, but the Earth's heroes were easily defeated, and then they listened to the village elders said that the once great hero Sam in his grave left a powerful magic weapon, so they visited Sam's grave found the time capsule he left, which has fish bait, books, Buddhist beads and summoning cards, the heroes according to the book, the use of fish bait to catch the monsters, with cards to summon Pikachu Thunder defeated them, the aliens fled, the world restored peace.

[chenzou]

Time Capsule Part II

Imagine the possibilities. It is a wooden fish often used in traditional religions, which is hollow in the middle so that it can be struck with a set of wooden sticks to make a crisp sound, and I tried to convert the sticks into a lock to trap the items stored in it, which is more suitable for my string.

Yuanbao is a good time capsule, he signifies wealth, in ancient times, it is cast from solid gold, corrosion resistance is very good, I am considering his cost therefore in want not to open made into a shell that can be opened.

[chenzou]

Time Capsule Part I Chen

Pokémon is a childhood memory of mine, and it has been with me since I was in kindergarten. Although this card was a replica obtained at a McDonald's event after I came to the U.S., I actually have a stockpile of close to 10,000 cards in my home in China, and this was almost completed as a child, so you can see that my allowance sacrificed a lot for me. Before there were any Pokémon related video games children almost established class status among themselves through cards! Who has a rare card will be revered. And many in the absence of pocket money to buy snacks when such cards can be traded as hard currency among children, almost like the status of the dollar as a savings currency among children, and can be bartered. Although growing up I found that I may have suffered a childhood investment failure resulting in my current lack of dollars and gold, because it reminds me that life also has its moments of glory.

You can see that he is a gorgeous guy, he shines with a mysterious luster, he is my old comrade, a steelhead that sold for thirty dollars. He has accompanied me on a lot of sea battles, and he has had a lot of success and a lot of success. My first target was an island reef near my hometown, where in September every year there are powerful yellow-striped amberjack as big as a bull, there are dark currents, no one on the boat dared to be careless, I used it to search in 30 to 60 meters of water, due to inexperience eventually I was careless, it was a heavy guy, the model of my rod did not match it very well, I lay on the deck exhausted and sunbathing, while the boat The experienced anglers continued their work. To my surprise after most of the day's search no one had too good a catch, the catches were almost all small fish, as I was about to head back I tried to make one last attempt but I suddenly found I couldn't retrieve my line, I thought I had hooked a reef in the sea but my line kept pulling out, I had hit the jackpot! I retrieved my little fish with all my strength, this tug of war lasted for ten minutes, the silver light flipped on the surface, this was my first prey over twenty pounds, I was so excited, it followed me to many waters after that, I also took it to challenge the bluefish off Long Island after I came to the US for graduate school, you can see it lost its eyes, but it was unbeatable, it was my mascot!

This is a bracelet with a Buddhist theme, because of its length it is usually placed in the hand to finish coiling. I used to like to collect some Chinese literary objects during my college days, I have many treasures including jade, amber, bone products, etc., but due to the convenience of travel I only brought it with me, it is actually made from the seeds of olives, I value it more for its subject matter than for the material. The eighteen Luohan are traditional Chinese gods and goddesses, which can be understood as the guards of Buddha. They have different expressions representing the joy, anger and sorrow of the earth, and it helps me to comprehend some thinking when things are not going well.

This is a book about traditional Chinese theater costumes, which I participated in editing during my university days. Although the main editors are teachers, it also contains my heart and soul, I am mainly responsible for data collection and picture taking, if you have a deeper understanding you will know that this kind of costumes are not washable, and it is troublesome to carry, there are too many small pieces need to split and organize, countless trips between the theater and the studio, how difficult it is to compile a book!

[chenzou]

Spatial Interaction Hypothesis

In fact, I do have some random thoughts on interaction modes, such as which are the forms of interaction we are currently missing. I am more interested in the overall development of the general trend, the current two-dimensional screen as the mainstream form of interaction is still no qualitative change, which is mainly caused by the way the information is transmitted, but as the speed of transmission is getting faster and faster, we can almost overcome the limitations of time, can we next try to cross the screen? In other words, break the limitations of space? I think this is also the inevitable trend and development direction. I was initially inspired by VR movies and AMD, because it puts us into a full space rather than interacting across the screen, in order to break the limitations of space on this basis then I thought how can we reproduce the real space in real time? Does this mean we can go to any corner of the world at any time in terms of audio-visual? Or even remote control of robots and the like will have greater development.

Then I would like to briefly discuss some of the current tools that are likely to be developed. First of all, the VR camera is too expensive to explore specifically. I first envisioned some of the more easily available tools. A depth camera is my first choice, and the best option I could find so far is the Azure Kinect Kit. This is a development tool in the field of deep vision, and I'm not deep enough into the code yet, so I'll look into its functional requirements here first. The best thing about it is that it can collect very accurate point clouds, what does this mean? We can quickly get the position and shape of objects in space, and it has RGB color mode to collect color information, which is currently envisioned to be paired with a movable crawling robot, and the program needs too much writing yet to be implemented. Here first do not expand the discussion.

It is envisaged that this step can only be described as the ability to capture spatial information, but it has not yet reached the perfect interactive effect.

[chenzou]

Butterfly

We tend to focus on some of the larger and more famous protected animals, but I think some of the more obscure animals are more difficult to protect. In order to make some more comprehensive protection, I chose a butterfly species unique to China, the Golden-spotted Beaked Butterfly, which is one of the most valuable butterflies, it is extremely rare and is the only one of the first class national protected animals in China. The golden-spotted beaked phoebe is a subtropical and tropical alpine species that inhabits the evergreen broad-leaved forest mountains at an altitude of about 1,000 meters, rarely coming down to the ground for drinking and other activities, so it is not easy to be found and captured. The butterfly originally had a good meaning of fidelity and rebirth, and someone once offered $100,000 to buy it.

I collected some recyclables, cardboard, wire, chopsticks, etc.

I tried to use the support of the chopsticks to make it possible to simulate flight, but I had a little problem with the accuracy of my punching, and the strength of the glue was a little unable to support the whole butterfly, which I tried to correct.

[chenzou]

I chose origami as the theme first, because it is imaginative in itself, but it is possible to keep at least one distinctive feature in a huge number of sketches so that we do not lose the theme, those folded corners. On this basis there are countless kinds of themes, because we are currently limited to one condition, but of course we can specify more restrictions to highlight the theme, especially in the human brain to stay in some stereotypes will help drive the theme, but of course it varies from person to person, for example, my mind to establish origami as the first element when the second thought of white, the third for children's fun, and so on. We need to make trade-offs according to the actual application, including the choice of color, material, shape and other elements, the more dispersed the choice, but often for a whole set of works the theme and the overall nature will become weaker. According to the requirements I tried the flat, three-dimensional and actual material vision, through the second feature can be classified in detail.