Сообщения

Report from the conference Lucene Revolution

Изображение
In early October I was able to attend Lucene Revolution , which was held in the hero-city of Boston. The conference was dedicated to open search technology Apache Lucene and Apache Solr . It seems to me that on habré in particular and the Internet in General these technologies is given undeservedly little attention. Let's fix that omission. I have long been engaged in these technologies, but could not imagine that the conference on this narrow specialized area can collect a number of speakers, participants and companies, the total number of participants was more than 300, more than 40 speakers. The range of implementations of data search technologies are quite impressive — a social network mikroblogerov services, CRM platforms, online shops, government websites, online and the University library, Dating sites and even projects in the field of bioinformatics. LinkedIn The largest social network of professional contacts contains more than 80 million user profiles, ...

PostgreSQL Asynchronous Replication + Pooling + Failover

Option is simple to understand asynchronous master-slave replication on Postgresql 9.1 database / > For the first time the task was solely implementing full replication for the first time and was written a mini-manual, which I would like to present here. For system replication Master-Slave used a combination of the the PostgreSQL 9.1 (DB) + Bucardo 4.5 (Replicator) + the PgPool-II (puller and failover) Bucardo Async Postgres replication system written in Perl5. The easy adaptability to Postgres and light reflected in the database configuration. Creates its database in which are written the replicated server, database, table, concluded in the leaves (lists). Used type of when Pushdelta (Trigger. One way master-slave). Changing the structure does not support. Works both ways, i.e. in the event of a temporary disconnection of the wizard when you restore, it will automatically "catch up" slave. A rough plan of installing bucardo o...

oEmbed. Doing a web service? Do and oEmbed

Изображение
Of course you've seen this effect when inserted in Facebook or myspace link in the post immediately begins to load some content from a website to which the link leads. And I saw, but didn't realize how easy it can be done oEmbed'ohms. oEmbed.com — in principle, this could be and finish reading for those who want to undertake independent mini-research. a Little theory oEmbed is an open format created to facilitate the implementation of the content of one web page to another. The role content can be photos, videos, links, or other types of data. Simply put, the use of oembed allows you to embeded content (e.g., YouTube video) when a user post a normal link to it on your site. The exchange of information, from the point of view oEmbed, occurs between the supplier and the consumer. A consumer wishes to show an embedded representation of third party resource on their own website, for example, it can be a photo from flickr or videos from youtube/vimeo. Th...

The art of positioning

Изображение
Imagine this situation: you and your team are going to launch/launched a startup, be proud of your idea, work day and night, and one day, returning home late at night to meet his old friend. There is the following dialogue: ... — What do you do then? Yes, we guys launched a startup, so cool! — Oh, cool! And what kind of startup? — Well, listen, I'll explain. In General, it is the directory with the music in which it can be estimated, likes and share it with your friends. And most importantly, everything is free! — Well of course. Okay, I went to the meeting. And you go and think why your idea was not appreciated. About the same happens when a user comes to your site and gets no response to the question of who you are, what you can do to help him and you are better than others. Next, I will try to tell you how to make a correct opinion about yourself and give examples. Basic principles of positioning The moment when the user goes to the site of yo...

How to start a search, or a few thoughts about crawler

ongoing threads about their own search engine So there are a few major tasks that the system must solve the search will start with the fact that a separate page is necessary to get and keep. There are several ways, depending on what methods of treatment You will choose in the future. Obviously, it is necessary to have a queue of pages that need to be downloaded from the web, at least in order to then look at them long winter evenings, if nothing better not come up. I prefer to have a queue of sites and their home pages, and local mini place that I will process at that time. The reason is simple – a list of all the pages that I would like to load just for a month can significantly exceed the volume of my big hard drive :) so I keep only what is really needed – sites at the moment is 600 thousand, and their priorities and load times. When you load another page, all links from this page should either be added to the local queue if they are within the website which I ...

Bruce Momjian, Marco Green, Fedor Sigaev and Maxim Boguk answered questions about PostgreSQL

Изображение
June 30, we held an online session of "Questions to the world's leading experts PostgreSQL". Within three hours, anyone could ask a question to one of the invited guests and get a comprehensive answer. The result is more than 40 questions, none of which went unheeded. Participants: Bruce Momjian (Bruce Momjian) (PGDG, EnterpriseDB), Maxim Boguk (Rambler, Masterhost), Fedor Sigaev (PGDG), Marco Creen (Marko Kreen) (Skype). Here are some selected questions and answers: the Question # 3 the kai kai : One of the unfortunate pieces in the work is the absence of design "INSERT OR UPDATE". In MySQL you can do this: "INSERT INTO users (username, email) VALUES ('Jo', 'jo@email.com') ON DUPLICATE KEY UPDATE email = 'jo@email.com'". Will there be something similar in PostgreSQL? the Bruce Momjian, the leader of community PostgreSQL, the expert of the company of EnterpriseDB: Standard (ANSI) method for such t...

Nested Sets + PostgreSQL TRIGGER

Task How convenient to make selections of trees of type Nested Sets, and not convenient to manage. How to udobnoposvetu trees types id->parent_id, but how inconvenient and expensive to use recursion with selections. It is clear that when using the modules for managing trees part of the problem is removed, but the process of working with the database not quite transparent, i.e. to modify data, we use some methods to change the location of the node in the tree,plus the transaction would not hurt. This inconsistency can be solved in two ways: the to be Used for a table of stored procedures, which combine both methods of update (insert, delete); the to Use triggers to avoid in General any non-standard methods of work; the First method is that when you change the table structure, we need to change the procedure, as well as betmaxcasino careful when working with a table that all changes to the data have passed through our procedures, not paramilitarism. The second method is some...