A year of working as data scientist

I want to give a brief overview of working as a data scientist for one year. I wrote several posts about my progress and now it is time to look back and see what was accomplished (though all of this is only a beginning). Maybe my story will help to motivate some people planning to work as DS.

• Started working in a bank on April, 17 last year;
• Completed machine learning specialization by Yandex and MIPT on Coursera;
• Completed most of cs231n course;
• Build a first pet-project: online recognition of handwritten digits. This project was well received and people still use it - the app was used at least 2000 times in the current month;
• Realized that working in the bank isn't interesting and rewarding, also SAS isn't fun at all. I changed my job and started working in a startup;
• Completed second session of ml_open course (here is a link for english version) and finished at the 5th place;
• During 2 months of working in startup I did only one task, and there were no definite plans for future. Also several crazy thing happened and I decided that I deserve something more. So I changed my job again;
• Completed two courses (kaggle and nlp) in Advanced Machine Learning specialization on Coursera;
• As a part of final task of nlp course I made a telegram chat-bot @amlnlpbot;
• Started taking part in official and unofficial meetings of data scientists;
• Continued working on my portfolio: https://erlemar.github.io/ ;
• Realized that there was only little professional development for me in the current company and decided to change my job once more;
• Tried to take part in machine learning competitions but without success. Currently I take part in a competition on Kaggle, where prizes are given for kernels with the most votes, here is the link;
• 
  

Thoughts:

• A year ago I couldn't imagine how amazing, motivating and useful would be ods.ai community :) It is possible that without it I would still stagnate in a bank;
• It is very difficult to find balance between practice and theory. I know that my theoretical knowledge isn't enough, especially regarding statistics and maths, but I have yet to have problems with these spheres at job, so currently I practice morel
• Sometimes I feel that most companies (excluding big or/and advanced ones) hire DS just to have them, while having little relevant tasks or not understanding their possibilities;
• Realized it is necessary to take part in competitions, even if it is only for experience;
• I suppose it is worth investing more time into learning DL. Of course there are many interesting tasks without it, but most of them are related to marketing;
• It is important to improve programming skills;

Plans:

• Pay a lot of attention to DL. Complete fast.ai course or some parts of it, then try to implement popular papers;
• Take part in competitions on Kaggle. Earn at least a silver medal;
• Maybe try learning R to understand why so many people praise it for data processing and visualizations as well as Shiny;
• Improve programming skills. Maybe learn Java/Scala for writing production solutions;
• Create 1-2 more pet-projects;

Some things change, others stay the same.

More that 1.5 years ago I have decided to change my career to Data Science and Machine Learning. Since that fateful decision I have been spending most of my free time on getting new knowledge and skills. This is quite fun, though as a result some other things need to be sacrificed.

There are so many things to learn: fundamental knowledge is necessary (like statistics and math), programming (improving Python skills and possibly studying a new language), analytics and ML itself.

Telegram chat-bot
Some time ago I took a course on NLP on Coursera and mostly liked it. On the one hand it gave me a lot of useful practical and theoretical information, on the other hand sometimes there was too much theory, also writing code in Tensorflow and debugging it is tough (thanks to this I liked Keras even more :) ).
The final task was to build a Telegram chatbot. The main functionality was finding an intent of the question and giving the appropriate answer. There were two main intents. If the question was related to programming, the bot should send a relevant answer on Stackoverflow. Or if the user simply wanted to talk, bot should be able to chatter. For a default implementation a trained chatbot model was offered.
And then there was an additional (so called honor) assignment - the challenge was to build and train conversational model by ourselves.
I have successfully done it, here is the link to the bot and the link to the github.
I won't bother you with technical details (if you are interesting, you can read more on github), I'll just describe the bot functionality:

• If you ask it a question related to programming (belonging to 10 most popular languages), it will look for an answer on Stackoverflow;
• If you send a sentence with word "weather" and city name, it will give a weather forecast for the next 5 days;
• 'tweet/twitter account_name' command shows the latest tweet by the user;
•  'today!' - shows current date and random fact about it;
• And, of course, bot can talk to you! Though it is kind of dumb :)

Working on this project was really fun and I learned a lot of interesting things - for example, how to host a running bot on Amazon EC2.

DonorsChoose competition on Kaggle

Founded in 2000 by a high school teacher in the Bronx, DonorsChoose.org empowers public school teachers from across the country to request much-needed materials and experiences for their students. At any given time, there are thousands of classroom requests that can be brought to life with a gift of any amount. DonorsChoose.org receives hundreds of thousands of project proposals each year for classroom projects in need of funding. Right now, a large number of volunteers is needed to manually screen each submission before it's approved to be posted on the DonorsChoose.org website.

So the fund has created a challenge on Kaggle to help building a model to automatically approve applications.
The prizes are awarded for the most voted kernels, so it isn't necessary to build a high accuracy model.
I have dediced to take part and here is my exploratory data analysis.

Other things
Sadly, I have little time for studying foreign languages currently... so I try my best to keep my knowledge: my phone system language is Spanish, I try to read Spanish and German news, still review flashcards in Anki and sometimes read manga in Japanese.

Last year I started reading indie books and some of them are true gems! Recently I have discovered "Chronicles of the Black Gate Series". This wonderful series takes place in the world, where humans believe that place in society is determined by your birth. Depending on place of your birth you could be a warrior, slave or even heretic. Thanks to reincarnation you can move to a better or worse place. There are multiple points of views ranging from teenagers to adults. Character development is quite believable. A lot of characters face a crisis of belief which results in strengthening of beliefs of some characters and changing or bending of beliefs by others. . Price which is paid for success - and sometimes it is too high! And speeches which characters give are really impressive and strike to the core. I really liked the series.

My results of 2017

My main accomplishement in 2017 was successfully changing career to Data Science. Doing it wasn't easy, but it was definitely worth it. Of course, there are many things which I have yet to learn, but this is a start and I already have a middle level position.

Also I have read a lot of books this year, most of them fantasy. The most prominent series were books by Brandon Sanderson about Cosmere. On the other hand I have more appreciation for indie authors and for writers of fanfiction - you can find real gems there.

Progress with machine learning

Half a year has passed since my last post. I have accomplished a lot of things:

• Completed Machine learning specialization by Yandex and MFTI on Coursera and cs231n course;
• Got a job in a bank;
• Finished my first stand-alone project on machine learning;

Machine learning specialization

This specialization covered all main topics in machine learning - classification, regression, clustering, nlp, recommender systemsand so on. I think that this is one of the best courses for beginners. As a final project I made a little site with sentimental analysis functionality - I have gathered ~25k customer reviews on mobile phones, trained a model and created a little site, where user can input a review and receive the result of analysis - whether the review is positive or negative.

cs231n is Stanford's course on deep learning. It was amazing and extremely useful.

Job in a bank

Finding a job, when you radically change a career if difficult, but after some time I was able to find a job in a bank. Now I work in a department, which works on cross-selling - trying to sell bank products to people, who already have some product of this bank.

I'm building a model which predicts probability with which customers will activate the card, which bank sends to them. And I'm doing it alone! So it is challenging and interesting.

Analysis and prototype are done in Python, but the solution in production will use SAS. So I have a lot of things to learn and explore.

Stand-alone project

In my spare time I try to learn as many new things as I can. There are many ways of doing it, one of them is building pet projects. I have spent several weeks on it, but the result was worth it.

Here is the link. The site looks like this:

This is a little site (also working on mobile), where a user can draw a digit and machine learning models will try to recognize it. There are two models: FNN written on numpy and CNN on Tensorflow. Both were inspired by cs231n. But models don't simply recognize the images, they are also trained on new images to improve their accuracy.

I collected the data by myself - I drew 1000 images, ~100 of each digit. Then I created the models and trained them on this data. This was my first experience in:

* Collecting, processing and storing the data by myself;

* Building a site and using Flask;

* Building models, which can be trained on additional data;

* Working with Amazon cloud;

* Augmenting images;

* Building new good TF structure for my purpose;

I have described the models and the project workflow on site, you can find the links in the navigation bar at the top.

Learning. Machine learning.

I spent most of the last several months on studying programming and machine learning. This is difficult in many ways:

• Previously I had only a limited experience with programming, and now I need it. So I decided to study Python, which is a great, popular and not difficult to learn. It took me 1-1,5 months to learn enough to start studying machine learning. I went through a great book "Automate the boring staff with Python", it was amazing. Then I read some useful articles, practiced on sites offering exercises and so on;
• I had to renew my knowledge of linear algebra, calculus and other mathematical stuff. This is really necessary for understanding of machine learning algorithms;
• Also I needed statistics and theory of probabilities. While I have already renewed my understanding of statistics when I was preparing for Lean Six Sigma exams, theory of probabilities was never my forte. So it required some efforts to study it;
• Of course analytic skills are also necessary.

And machine learning by itself is very challenging. There many areas in machine learning and they differ from each other. Of course, while programming you could use libraries for fast and easy use of many algorithms, but if you need to reach better accuracy and performance than average, you have to understand how these algorithms work, what is the math and logic behind them.

Starting machine learning is easy thanks to a variety of guides and learning materials. Understanding it is much more difficult. I hope I'll be able to do it.

Meanwhile I am in process of building my portfolio to show my skills and knowledge in this sphere.

Summer 2016

Well, trying to write more regularly proved to be difficult, especially when you have your hands full. Some things which I'm going to describe may deserve separate posts... I'll see what I can do about this later. So these are the things I did in the past 7 months:

Leaving old job:

I left my job, which I had for 2 years and 8 months. The decision was long overdue, but better later than never. The long project, on which we worked, was completed (as much as possible), the prospects were unclear and I didn't have an increase in salary since I have completed my internship. So it was time to move on. Two weeks were spent on healing a cold, which I caught accidentally. Immediately after that I got an offer for a position of analyst in a company in Skolkovo (yes, that Russian innovation center). After a couple of interviews and another two weeks I got this job. What happened there is another topic.

Self-improvement books:
I have read various self-improvement and business books in previous years and several more this one. IMHO, most of them can't be used by usual people. It's not that such books are bad, they just work only in certain conditions. If this is the book's target - it is okay, but when the book is aimed at bigger auditory... The reason is that the technics and ideas described in the books can be applied only in limited area of expertise - for people with certain cultural background, in certain countries, maybe in certain work spheres or even only in certain situations with certain people. Often it sounds like a millionaire telling a poor man how to live and work. Of course there are some useful things in these books, but they can be usually summarized in a couple of pages. Also many books blatantly use other authors' ideas. So I think that it is enough to read several suck books, take the key ideas and use them. The books better be good such as Psycho-Cybernetics, Seven habits of highly effective people or even Meditations (by Marcus Aurelius).

Reading Bible:
I'm atheist. And I don't like a principle of "didn't read but criticize", so I decided to read Bible (because Christianity is better known to me than other religions). I took an audioversion of King James Bible and listened to it. I went through Old and New Testaments and decided that my atheistic believes only strengthened. Here are some points which I didn't like. Some of them are borderline jokes, some are my personal opinion, in some points I could be wrong; in any case I have no intention of insulting anyone with these words:

• Old Testament was full of violence, adultery, infidelity, stupidity and so on;
• New Testament was better, but still had a lot of ideas, which I don't like - for example forgiveness of everything bad which is done to you, propaganda of suffering, humble and insignificance of life in this world, submission of women to men and many other ideas;
• A lot of internal inconsistencies;
• Bible was written when people had very limited knowledge of the world and wasn't update since that;
• There were other people besides Eva and Adam. How did they appear?
• Exodus story was very cruel. Considering it was God who made Pharaoh not willing to let the Jews go.. and then punished Egyptians for that. Well, there are many cruelties in the Old Testament;
• Jews make the same mistakes again in the Old Testament. For example Moses and Exodus. Jews get angry on something, Gods punishes them or wants to punish them, Moses pacifies him... But Jews start irritating God again;
• Why was Judas betrayal necessary? Why Jesus cursed a Fig Tree? How people can understand that they meet the Beast and not the true God?
• And many other things...
• In the end after having read the Bible I continue living as I have lived before: trying to be adequate and honest and not hurting other people unnecessarily;

Lean Six Sigma:

I did several more things for self-education. First of them was learning about Lean Six Sigma. This is a methodology of quality management. How to eliminate variation of the process (so the results are better and within the necessary limits) and waste (remove unneeded steps from the process). It is a solid and useful methodology used by many companies. And it heavily used statistic methods. I heard about it from my colleague and decided that I want to learn it. There are several levels of proficiency with exams for each level: Yellow Belt (beginner), Green Belt (medium), Black Belt (team lead), Master Black Belt (even better than Black Belt). After two months of study I was able to pass Green Belt Certification! I hope to be able to use these skills and knowledge in my job.


Touch typing:
One more thing which I decided to learn is touch typing - fast typing on keyboard without looking on it. I can type fast enough, but I have to look on keyboard sometimes. As I need to type huge amounts of text sometimes, I need to be able to type fast and efficient. At first my speed dropped dramatically, but after two months my touch typing speed in Russian became faster that typing speed which I had before (almost 400 symbols per minute)! Now I'm going to learn touch typing in English;

New job:
Now about my new job... I'll start from the end - last Friday I left it (5,5 months of working) and was very glad to do it! What went wrong? Almost everything. This is a really long story, I'll tell it some other time, now I'll describe only main points.

• So, I was offered a position in this company. Of course, I decided to look for information about this company. I have found this (text in Russian). The company has several contracts with Pension Fund of Russia. The company and some other companies with the same leaders have contracts for 300+ million rubles... Well, it doesn't directly concern me, so not a big deal, but still a thing to ponder over;
• So, I got the job. The first task? "Let's see, we have a big client with a system of internal budgeting on SAP BW/BI. You have 1 week to understand how it works and a create a first prototype/design of dashboards for it!" It took me one day more than planned, but I delivered it! And heard about it only a couple of months later...
• Then I spent almost two months writing documentation for this budgeting system. It became boring very soon. But there was a possibility of something more interesting and I waited for it. And when I finished it I had to change formatting of all documents and had to stay until late due to poor organisation and communication;
• Then a new project began. Developing a web-reporting system for the budgeting system (with the first description done by me previously). The team: junior front-end developer, senior back-end developer, analyst-tester-pm (me), ux-designer, outsourced designer and project leader. And two bosses (who lead the company). It was a mad project, describing all of it would be too long, so just several points. The planned length of the project was ~3 months. Design was thought to be done in one month, but it continued till the end of the project, because the scope of design was revised several times. The scope of project was changed several times. Client didn't take any part in this project. Internal reviews by bosses were full with criticism and unfounded expectations. Project leader and me didn't have any idea about web-development so we had to use a trial and error method. Also I had no experience in managing. So it was crazy. When it was acknowledged that it can be installed for clients (despite a number of bugs and unfinished functionality) I resigned;
• Company's bosses preferred a stick and carrot policy. Only without a carrot. They didn't value human resources and were sure that their decisions are almost always correct. Now wonder there was s serious staff turnover;
• So I'm really glad I left it at last. Now I can have a rest and look for some interesting job;

HTML, CSS, JavaScript:

While I worked on the aforementioned project, I decided to acquire new skills in relevant spheres. I learned HTML when I was in school and it didn't take a lot of time to remember it. CSS took some time, but I needed only basics - just to be able to understand how it works. JavaScript is another level for me. I don't know programming languages so I just repeated the things which I found. I think I'd better put it on hold for better times :)

Japanese:

I had little possibilities to actively learn Japanese. So I just read raw manga and random articles in Internet, watched anime and so on. Good news is that I feel progress, bad news is that I need to really concentrate on learning Japanese to reach a decent level. I hope I'll have time for that.

Spanish:

More attention was paid to Spanish. I read a lot of news articles, watched news video and tried to immerse - like switching my phone to Spanish. Also I went through La Sombra del Viento one more time, translating unknown words. I even tried to write something in Spanish, but it was really bad, It seems more practice is necessary. Will try to do something about it.

Big Data:

Recently big data became a popular trend. Maybe too popular. But the fact is that many big companies use big data analysis to increase their profits. And it is interesting. After all I'm economist by education. So I want to try myself in this sphere. Two main areas of knowledge which I have to explore are mathematics and programming. I had experience with mathematics in University - linear algebra, mathematical analysis, theory of probabilities, econometrics, statistics and so on. But I didn't have success in all of them, so excelling in this area will be challenging, but it would also prove that I became smarter (at least I hope so). As for programming - I have little experience with it, so it is like a black box for me. But the more difficult it is, the more interesting will the learning be!

So I lot of things happened to me in the past months and I learned a lot of new things. And I hope to learn much more soon :)

Several situations at work

The project I was working on for 2,5+ years is ending.
Today I was looking through my mail and discussions in various messengers and have found several funny (at least from my biased point of view) situations, which I'd like to share. They show a difficulty of producing a big project with a lot of developments. The situations are modified a bit to be less personal.
Consultants (C), testers (T), Developers (D). managers (M).

3 developers and 2 consultants are discussing a report. User's expectations of the report are unknown, but all the participants of the discussion have different opinions on how the report should work...
______________
M: This video "The expert" (about 7 parallel lines) is really funny! Clients are so crazy and unreasonable sometimes!
C, T, D: Should we tell them that mirror may show them a similar thing? :)
______________
T: A bug! I have found a bug! Here is it!
D: Let me see... several minutes of hitting the keyboard - there is no bug, I don't know what are you talking about!
T: But it was there! *pouts* I have no proof, but I sense that it was your doing!
D: I won't tell anything :)
______________
C: You test bad! The quality is contrary to our expectations!
T: Could you tell us your expectations at least now?
C: You should test better!
...
several days later
T: Here is a bug, one more, another one, also this and that...
C: Stop it! There are too many bugs already! :)
______________
C: Why additional document for this document isn't created? I told you to remember about these cases! How are you testing? Must I retest everything myself?
T: Well, here is the same example of 2 weeks ago. You told that time, that it is correct. And the responsible team checked it.
C: ...
______________
T: We need to show this functionality to users now, so that we can fix all the problems in time.
C: Nope.
...
3 months later.
C: Today is the training session on this functionality and nothing works! And only 1,5 weeks are left before UAT.
T: ...
______________
C: We need you to create test examples for all functionality of the system, check and fix all issues and prepare master-data for all test scenarios. You have two weeks.
T: But this is impossible!
C: You are telling us that you can't create several simple objects in the system?
T: We can, but...

C: Then do it! You have clear requirements after all. Ah, almost missed one more thing: write manuals and train users. And don't forget to be motivated by having an opportunity to work on this awesome project!
______________
T: If I enter "*random strange text*" there will be dump! And I you copy the object, delete the new object, copy this deleted object and repeat this exactly 6 times, there will be another dump! Also if I use a price of a milliard $ for a gallon of oil, there will be an overflow dump! And if I select this line and delete it with a keyboard shortcut, there will be a strange dump! And if I..
C,D: Shut up, crazy pervert! How do there abnormal creatures even appear? :)
______________
And the last one:
- No one likes testers!
T: It isn't true! We like ourselves!

The end of 2015

I wrote nothing for almost a year, because I had a tough year: my grandmother was slowly fading away in her bad for 10 months. It was hard enough both physically and emotionally, but it was doubly worse due to her spiteful nature. It is quite exhausting to take care of the person, who considers you almost an enemy, isn't it? Now this is over, but some time is necessary for a complete recovery. During this year I maintained my mental health with language learning and some games, and this is what I have to say:

• Last year saw a rise of so called "idle games". Previously there were some games in this genre: Progress quest - RPG, where you could create a hero and then only to watch his activity without any interaction; Godville - the game, about which I wrote earlier, where players have limited possibilities to interact with hero and several other games. But in 2014 Cookie Clicker appeared. Basically it is a game where you... click a Cookie.

Yes! You just click this cookie and gain money. Money can be spend on the upgrades - like getting more money with each click and, more importantly, on earning the money automatically. The main feature is that money continue to be earned even when the game is closed = idle. Also many games have a so called "prestige system", when the progress in the game can be reset to get some additional bonuses. There are big upgrade trees in some games, the games themselves could be really different But the core is the same in most idle games - players click something to earn money, which are spend on upgrades for earning more money. And money can be earned with the game closed. That's it! Sounds strange? But these games earn there share in the market. At first they appeared as browser games on Flashplayer, Unity, HTML 5 or something like this. Then they became popular on mobile devices, for example, "Tap Titans" have more than a million votes and 5-10 million plays.

And idle games earned their place on Steam. So why such a popularity? Firstly, they are really addicting! Try, if don't believe... You click - and there is an immediate reward. Figures grow fast. Achievements can be earned. And so on... Secondly, they can be played even with little free time. Thirdly, after some time most games turn from micro to macro managing which is also interesting.

I spent a lot of time on them and it was fun. But I'm glad, that I was finally able to drop them, as they could consume a lot of time.

• As a part of deepening my knowledge of Japanese I continued playing visual novels. One of classifications could be based on the presence of gameplay in the game or it's absence. In some games gameplay is quite unique and captivating with no need to repear something. But some companies produce games with an insane amount of grinding! I think that some MMORPG have less grinding. For example, let's take a game "創刻のアテリアル" with a card-based battle system.

There are heroes which can receive experience and level up for bonuses? Cool! But there are 33 levels and heroes can't be leveled to max level in one game...

Cards can receive experience and grow? Cool! But cards receive only 1-4 points of experience in each battle (100 points are necessary for a level up), and cards can have up to 9 levels (so up to 900 battles to max one card)...

Some cards can be crafted? Cool! But for one high level card several cards of a lesser level and tenths of ingredients can be required...

And this isn't a full list! And this isn't the most grindy visual novel I met either.

On the other hand I have read a visual novel without gameplay called "Dies Irae". It was fantastic. Epic story with really different routes; many unexpected twists - you never know who will live and who will die, main hero don't always win and so on; wonderful art; memorable characters; and author likes to you quotes in different languages - "Faust" by Goethe, some French phrases, German hunting songs, Latin requiems and many others. I was enchanted and fascinated by the story and definitely will read it once again.

• Speaking about language learning. For Spanish I have found an excellent free audio course. The essence is making you not to just learn a language, but to think. The form of the course is a dialogue between a teacher and a student. At first we are shown that there are a lot of words in Spanish which can be guessed from English, or more precisely from Latin. Then we learn general structures. verb tenses and other things. The course is 14,5 hours long and gave me more than several months of self study.
• And once again I realized or remembered that learning a language requires an immersion. So now my mobile phone is in Spanish, my browser is in Spanish, I follow Spanish pages in my social networks, read in Spanish and even try to think in Spanish. The progress can be really felt.

Well, there were other things in my life, of course. But that's it for now. I'll try to write more regularly from now on.

Learning Spanish - week 8

This week was mostly spent on learning through reading news. Main purpose was acquiring new words. I found two good news portals and read 2-3 articles every day on various topics.
For this I used a program Foreign Language Text Reader (FLTR) (the screenshot if from it). It is very similar to LWT, which I described ~a year ago. For me this program is even better than LWT; of course there are some disadvantages, but they aren't really important for me:

• Though it is local (now web) I can use it from several computers by placing the program in Dropbox;
• No audio support - don't care, can use my local media player;
• Little statistics, just word count - anyway I don't use it;
• Can't test words in the program - don't care as I use Anki;
• And for for benefits. It is fast, free and supports all languages;
• Flexible template customizing, which allows to export data in a file, which can be imported to Anki;
• Easy to manage data - texts and words;

The only problem of all these programs is word forms: different forms of a word are analyzed as different words.
It is understandable - each language has it's own rules of words' changing. It would be necessary to program separate logic for each language. And even in that case there will be problems - in many languages one word can belong to one of several parts of speech depending on context (for example, "I filed the cards in alphabetical order" and "I order new clothes every year").
I almost accepted this inconvenience, but by chance I found an elegant solution... for Japanese. You input a verb and the program generates all its forms. The result can be exported to FLTR. This way all the verb forms will be marked and translated.
I wondered how was it done - maybe the program uses functionality from some website and shows the result. Maybe I could use it for Spanish. So I looked into the code - it wasn't what I expected. The author just described grammar rules with code: he coded the rules of verbs' conjugation for each possible case. No luck for me.

So now I just make notes for new words and don't spend time on making cards for different forms of known words.

So two months of studying Spanish passed.
During these days I was able to build a good foundation for studying Spanish language. I know enough grammar and words to read news on random topics with a moderate comprehension. Now only more immersion and practice is necessary. Several next weeks will be spent on reading news and articles. Then I'll read more books. And on one of my bookshelves "Cien años de soledad" is waiting. The journey to the depths of Spanish language is only beginning...

Learning Spanish - week 7

Finished this book. It was really interesting: intricate story, memorable characters, nice ending and so on.
Listening to the Spanish audio book while reading English text is really useful. I learned a lot of new words and phrases. Also i was able to notice flaws in translation: errors in certain phrases, missing sentences, misinterpretations and others mistakes. I hope one day to read this novel again and in Spanish.
In the following week I am going to try read newspapers. I have already tried it - rather difficult, but definitely not impossible.

Learning Spanish - week 6

This week I also practiced Spanish mostly by reading parallel texts while listening to audio.
I found a good site with several ready to use parallel texts with audio. It's a pity that only beginner and intermediate texts have audio. Nevertheless they are quite useful.
Also after acquiring some foundation and immersion I decided to try something more challenging. So I found the Spanish text of "La sombra del viento" and it's English translation (The Shadow of the Wind) as well as Spanish audio-book. This novel is a worldwide bestseller. It's quite big for a language learner - ~300 paged and 19+ hours of audio. But the more the better :)
As of now I went through the first third of the book. Naturally at first it was quite difficult, but the process is becoming more smooth and now I'm beginning to enjoy the book. This way of learning is quite fun.

Learning Spanish - week 5

During this week I tried to immense in Spanish more. Again method Listening-Reading was used.
I decided to try Harry Potter again as this book can be easily found in many languages. So English and Spanish ebooks and Spanish audio book (this was tricky to find, but i succeeded in the end). I have listened to this audiobook while reading the English version. Sometimes I have read Spanish version to check whether I have heard certain words correctly.
This was really effective so I'm going to continue using this method for some time.

The results of the year

2014 year is coming to the end. Many things happened during this time in the world, be they good, bad or something else. There were also events, changes and new things in my own life. So it's good idea to remember what have this year brought to me.

• Realized that I like learning languages. Not only it grants access to a whole new world of possibilities, not only it helps to train brain, but it is fun by itself. For example, there is a strange and unknown text. At first it is like a cipher text. But then you begin to unravel in step by step. Of course, there are some known words. Then some loanwords can be understanded. More words are guesses from the context. Inevitable, dictionary is used for unknown termins. And now you know what this text is about. You continue reading texts this way for a long time until something changes. You don't need the dictionary anymore. Sure, the comprehension rate of the texts isn't 100%, maybe it is just 60-80%, but it doesn't matter. If you see a sentence like "Evil pirates and their dastardly deeds should be punished!" and you don't know the meaning of "dastardly", this lack of knowledge doesn't impede your apprehension of the sentence. And also a general meaning - a bad characteristic - can be derived from the context. This means a quality change in the knowledge of the language and it inspires to continue studying it;
• Experienced more deeply the fact that it is very difficult to change a person's point of view and his opinion on the matters. Especially if this is convenient for that person. The person can be your relative, close person, friend, colleague, stranger with whom you have to communicate or someone else. The first problem is that most people don't like to have direct conversations. They prefer to be angry, disappointed, stresses by themselves or tell about these problems to other people, but not to talk with the one who causes these problems. Mostly this course of actions ends in quarrels and possibly break-ups. The second problem is inability to listen. It is much easier to speak, to impose one own views or just to ignore the words of others. Another problem is the people prefer to endure the problems rather than find solutions to them. Of course there are much more problems that those which are written here. But in the end the result is that people often stick to their ways as long as possible. From time to time - they do it too long and make the problems irreparable. And this is sad;
• Acquired the confirmation of the idea that the fact that someone is your relative can be a foundation for relationships between both of you, but it can be not enough to maintain them. If the relationship is one-way -  good things are done only in one direction and bad things are done in the opposite direction, then it can be better to cut these ties off. Or they will continue and will bring only pain and problems;
• Making big changes is really difficult. It is much better to stay in the comfort zone. It is more simple to stay the way you are. But one should remember the meaning of the words of Lewis Carrol "we must run as fast as we can, just to stay in place. And if you wish to go anywhere you must run twice as fast as that". If you want to improve, to become better, to try something new, to reach success you have to change. This isn't easy, but it can give big rewards. I tried different ways of beginning a change: little steps which steadily brought me to the target or a sharp (and in some cases painful) jerk. Sometimes I failed. Even when i was able to change I felt the urges to stop it, to revert back. But when I was able to meet the target, the success inspired me to do more. And it was nice;
• Health is really important. A little health problem can cause painful consequences later. Little early care can prevent it. Some work on the health can bring big positive effects in the coming years. Health is with us for the length of the life, so it better be good. So why not improve it? So I started active exercises. Hope I'll be able to do it long enought to have some results;
• Once more remembered that I like reading. For several years I have been reading business books, manga, visual novels, textbooks, articles and so on. A lot of time was spent on learning languages. Oh, of course I read normal books, but they got buried in the flow of other things. This year I made a break and tried to read something good and big again. And I was able to remember the joy of it: imagining the world and trying to live in it, following the heroes and empathizing with them, forgetting the things happening around me, wishing to continue reading, coming up with an alternate course of actions or a continuation of the story and so on. As a result I have read more than 40 books and some fanfiction. It can amount to 5-8 mln of words, I suppose. I hope to continue reading a lot in the next year;

Learning Spanish - week 4

This was the last working week in 2014, so I was busy. Sadly it negatively impacted the time which I could spend on learning Spanish.
Mostly I just repeated the lessons on Duolingo.

Also I tried reading some Spanish news. At the first look it was like "A mysterious text with several known words". Then I tried to use my brain more and realized, that it's not that bad. Surely there were many new words and sometimes unknown grammar, but thanks to the title I knew the general content. And many words could be just guessed :)

Next week I'll have more free time, so I'll try another method of studying.

P. S. A couple of days ago I found a wonder full game - godville. It is a text RPG. And it is almost a zero player RPG - this means you can't influence most of the things which happen. The hero fights monsters, earn money, buy equipment, drink in taverns and writes entries in his dairy about each action. This is really fun :) In fact with levels you (the god of the hero) receive some possibilities to take part in the game more actively. And you can encourage or punish your hero. Or just tell him something. I liked this game!
And I found out that the original version is in Russian and it started 7 years ago. So I registered in the Russian version and began this fun adventure.

Learning Spanish - week 3

Today I finished Spanish (from English) tree on Duolingo!
This was quite challenging. I learned a lot of Spanish words and some of its grammar. Due to high speed many things weren't remembered good enough though.
For all it merits, Doulingo has also disadvantages - grammar isn't explained adequately. And for many grammar points there is no explanation at all. Also the balance of exercises isn't really optimal I think - I'd prefer more exercises of translating English into Spanish.
But well, you can't hope to learn language from one sourse. So I also used additional sources to study grammar. The question is what to do next. I suppose I'll do the following things:

• Review the spanish tree (by using "strengthening skills") and translate articles in "Immersion" section;
• Begin duolingo courses in Spanish for learning English and German. The former will provide more opportunities to practice translation into Spanish, the latter will give an additional benefit of strengthening German;
• Try to start reading Spanish news;

Learning Spanish - week 2

This week I learned more about verbs.
Bad thing: there ~17 tenses in Spanish.
Good thing: only 5 or 6 of them are commonly used.

Bad thing: depending on the person (1st/2nd/3rd, singular/plural, casual/polite) verbs can have one of 6 different endings in each tense.
Good thing: at least this allows to drop personal pronouns. For example: "I am tall" can be written as "Yo soy alto" or just "soy alto".

Bad thing: there are ~200 commonly used irregular verbs in Spanish.
Good thing: most of them can be classified in patterns.

As any language Spanish has easy and difficult points. It just takes time to learn them.
For now I have an open point about the way of learning the conjugations.

I still remember how I studied the conjugation of irregular verbs in English language.
I was somewhere in 1-3 grade and had a private language tutor. She made me drill all the irregular verbs one by one. It was boring as hell and took a lot of time, it was repetitive and non-inspiring. But as a result even now I remember forms of most verbs by heart. So it was really effective.
But I'm not sure that this way will apply to Spanish good enough, as there are much more forms in it.
For example let's take English verb "to speak" and see it forms in present and past:
speak, speaks, spoke... and that's it.

Now the Spanish analog - "hablar".
hablo, hablas, habla, hablamos, habláis, hablan, hablé, hablaste, habló, hablamos, hablasteis, hablaron.
And this is a regular verb!

Now you get the idea.

So at first I'll try to learn just patterns.

Learning Spanish - week 1

So I began learning Spanish last weekend. It was mostly Duolingo. ~700 words are in the vocabulary. Not sure how many of them I really know :)
In fact a lot of words weren't really new.
Some words are exactly the same as in other languages, for example "the animal" is "el animal" or "библиотека" is "la biblioteca".
Some words are almost the same as "der Pistole" and "la pistola" or "the calender" and "el calendario". 
Spanish, as many other languages, has many loan words, so this makes things easier.
Also in some cases things are simpler due to the endings - the world ending can change the gender the word, so that you need to learn only one word and not two:
la gata/el gato - the female/male cat
la hermana/el hermano - the sister/brother

But, of course, not everything is easy. For example, Spanish has more pronounce: they distinguish between "him" (lo) and "to him" (le). Also verbs have much more endings. In some cases word order is different.

During this week I learned several hundreds of most common words and the grammar for basic conversations. Next week I'm going to concentrate more on verb forms.

Anyway, learning a new language is a really interesting experience :)

New challenge is set - learning Spanish

It is rather sudden, but yesterday I decided to start learning Spanish. I wanted to take a break from studying Japanese and try a new language.
Spanish is widespread (in TOP-3 of world languages), interesting and not very difficult, afaik. Spain has a rich culture, so there are a lot of things to get known.
Anyway learning foreign languages is a good investment in many ways.

When one starts exploring something new, the first question is "How to do it?" Currently there are many ways of learning new languages:

• Many organizations and private tutors offer good education;
• The number of textbooks is impressing;
• Already many learning systems exist, and new appear frequently;
• There are many sites which offer information or courses;
• Communities (like lang-8) offer people a possibility to help each other on the way to the new language;
• And so on;

It is easy to be overwhelmed by the possibilities and one can spend days or weeks on choosing the "right" way or trying one idea after another, searching for the best one.

I decided to start with 2 resources:

1. Duolingo. I suppose this site doesn't need introductions. While it has some drawbacks, it's very good at the beginning and lower medium levels, I think. It provides enough exercises so that topics and grammar points stuck in one's memory, it offeres training in pronunciation and various practice in the form of immersion;
2. The second one is a good textbook called "Spanish An Essential Grammar". I used a book from this series while studying German and it helped a lot;

I suppose these two things will keep me busy for a month. Looking forward for the new knowledge!

The autumn is ending, was there anything meaningful?

Once again the season is going to change. I think it is a good time to draw a line and to summarize what was done.

Japanese studying:

• I decided to try reading another visual novel in Nasuverse (the shared setting of a number of TYPE-MOON works like "Fate/Stay Night) - 魔法使いの夜 (Witch's night). Didn't regret the spent time. Captivating music, interesting characters, good plot, nice atmosphere, fine art - all of this resulted in a great work. The story is about Aoko Aozaki - a wizard apprentice, her teacher, her sister and a quite boy from a rural location, who wandered into a dangerous territory. Took me around 50 hours of reading. Maybe I won't read it again, but it will be remembered;
• Then I decided to try VN with RPG part, because... well just like playing games :) So next were three visual novels by Eushully: 創刻のアテリアル (Arterial of Origins), 戦女神 0 (Battle Goddess Zero), 戦女神VERITA (Battle Goddess VERITA). The first one is about part of a city and people in it moving to a different world, where main heroes manifest unique abilities and fight monsters to return home. There are three routes: siding with demons or angels and staying independent. Story isn't very original, but still quite nice, so it is possible to enjoy it. The second game is placed in a fantasy world. The hero is a follower of one of gods and after series of adventures loses his body and his soul dwells in a goddess's body (hence the title). A lot of fighting ensures. The narration was quite interesting. And the last piece is the continuation, which is difficult to understand if previous parts weren't played. The gameplay for all these games is quite good, but sometimes repetitive; there is one major problem though - a lot of grinding is necessary to achieve the full completion. And the games should be completed more then once for certain things. And it has diminishing returns to time spent. It took me 100-140 hours for the first one, ~80 for the second, ~60 for the third one. Suppose that 10-20% of the first two games and 30-40% of the second one aren't done;
• While reading VNs I also decided to improve grammar skills. Used series Kanzen Master (3kyuu and 2kyuu). Understood that I have a lot to improve. Will certainly try again next year;
• Starting to read light novels now. A couple of first ones weren't able to keep me interesting, so now moving to others. I use Anki with Yomichan plagin for reading (see the screenshot). This way looking up new words is easy, and adding them to the deck is done in one click. I hope this will be a step to success, as extensive reading was that thing which greatly raised my English skills (of course good foundation played a very important role);

Some other things:

• Spent a week in a hospital due to a little operation. Hope not to stay in a hospital once again. It's a kind of experience one doesn't want to have;
• After 1,5 years of working our project is coming to an end (1,5 year for me, but the start date is May 2012). We are developing a SAP trading system for one of Russian oil companies. In fact it is a first SAP trading system, so most was developed from scratch. My main occupation was manual testing: alpha, beta, perts of functionality, pre-UAT, scenarios and so on. It's a miracle for me how we managed to finish it. Maybe in future the thing we created will become a new standard module in SAP. Anyway, the project is ending by the New Year's Eve. A new project or a new job will follow;
• This autumn I read a lot. Maybe I starved for good reading after manga and months of studying, maybe I just got my hands on good things. Today I finished the 13th book of Dresden Files. A detective story with supernatural staff is an entertaining thing in itself, and these series offer good humor and some interesting insight into American culture. And the wiseass, deadpanning hero is quite fun;
• Also I read some more funfiction. "He Who Fights Monsters" is a funfiction based on "Rosario + Vampire" manga, which is good itself. The manga is about human finding himself in a school for monsters, where he finds foes and friends, who protect him (until he becomes a badass). But what if there are no friends for him? What if he has none to help. Then he can only fight back as a human. A dendroid attacks? Burn it! Golem? Use water under high pressure to cut it... And so on. A dark, but interesting story;
• Another one was "Victory at Ostagar". Funfiction based on the game "Dragon Age". One key point at the beginning is changed, and the things become really different. What if Wardens make the light signal in time, so that Loghain can attack the darkspawn in time to win the battle? As a result the King is alive, the army is in place, the Wardens can act with the crown's help. But as the King is alive, his correspondence with Orlais continues... Various characters have more brains, the story has more twists. One could say that the resulting plot is better than in the original. And the story is more than 1,1 mln words long! It's almost 2,5 times longer than "Lord of the Rings"! It is organized in 5 volumes with illustrations here.
• The last one is "In Flight" (760k+ words) - a crossover between VN "Fate/Stay night" and manga "Sekirei". The events of the "Sekirei" are places in the Nasuverse. The main hero is Shirou and the magic system is like in other Type-Moon works. The tournament in manga is really kind compared to the Holy Grail War, but it still sword some serious and bloody scenes. The crossover makes it darker and serous, more interesting. Most of the tale follows the manga, showing how differently can Shirou behave. Good humor, developing characters, interesting story and other aspects of this work made me like it so much, that I read it twice :)
• And while reading so much in English I decided that it would be good to make this pleasant time spending also useful. So new interesting words are written down for future learning. Now this list nears 500 words, and this will enrich my vocabulary I hope.

The last day of the summer - time to check what was done

The summer is almost finished, so it's a good time to think whether it was meaningful or meaningless.

Considering studying languages:

• I was able to finish reading "Fate/stay night" last weekend. I think it took me 1-1,5 hours every day, so, yeah, it was a very long novel. It was really fun and provided a lot of useful words and expressions - maybe more then a thousand;
• In late July I tried to take a test to determine my Japanese level and the result was Pre-Intermediate. Not bad, but I hoped for higher figures. I think the main problem is that I never tried to prepare for such tests - at the beginning I learned necessary grammar and vocabulary, and then I just studied the things which I encountered. So for several weeks I drilled grammar - there are heaps of grammar points about which I had no idea. Also I specifically tried to acquire as much words from JLPT sheets as possible. So I'm going to tackle this j-cat test this week again and hope for higher results;
• Going for another visual novel now :)

On other things:

• I usually don't read fanfiction. But somehow I came upon "Hill of Swords" - a crossover between visual novel "Fate/Stay night" and light novel "Familiar of Zero". Quite a big story (~350k words)... and it is just incredible. After several chapters I was completely hooked and was almost unable to stop reading it. The plot is based on "Familiar of Zero", which has a good plot, but it's completely wasted by dumb and silly characters and tons of fanservice. The fanfic fixes the problems by swapping the main hero with the one from "Fate/Stay night". As a result narration is greatly improved, characters are more reasonable, fanservice is more adequate. Now I'm rereading it with slower speed to savour it; if only there were less typos >_<;
• At last read "Stranger in a Strange Land". It was interesting to read, but left mixed emotions as the ideas are aren't really close to me;
• I remembered that many years ago I read some fantasy book and was able to find out that it belongs to series of four books named "War God" of David Weber. So got them, they were fun, though didn't give any useful thought after the finish;
• Tried to read "Chronicles of Thomas Covenant", but my determination completely failed in the middle of the second book. I just can't approve of such a hero, he is too different from me. This series increased the list of books, which I dropped without finishing them. By the way, this list have less then 10 items;
• Currently reading "Dresden files" - quite interesting and entertaining;
• There were many frictions at my job - I almost threw it up, but one way or another it became more reasonable and peaceful; so continue it for the time being;
• Got some new health problems, hope to be able to fix them soon;
• After 6 years of wearing the same glasses I changed them at last. The first impression: I see world in HD! Well, with these glasses my effective eyesight is ~1,5 dioptres, but it is twice as good as in previous glasses. Anyway, now the world is much clearer for me :)