How to avoid data duplication in my ruby ​​database on rails

Question

I have snippets of data from another site and saved in my database, which works fine. However, anytime I update my application, the dumped data duplicates itself in my database. Any help would be greatly appreciated. Below is my code

require 'open-uri'
require 'nokogiri'
doc = Nokogiri::HTML(open("www.example.com"))
entries = doc.css('.block')
@entriesArray = []
entries.each do |row|
    Scrap.create!(                  
    title: title = row.css('h2>a').text,
    link:  link = row.css('a')[0]['href'],
    day:   days =row.css('time').text)
    @entriesArray << Entry.new(title,link,days)
end

Answer 1

You can use the check modelto increase the error when create!, if any check failed.

class Scrap < ApplicationRecord
    validates_uniqueness_of :title
end

And you can also use the method to create a new record only if it does not exist in the database: first_or_create

entries.each do |row|
    title = row.css('h2>a').text
    link = row.css('a')[0]['href']
    day = row.css('time').text

    Scrap.where(title: title).first_or_create(                  
      title: title,
      link: link,
      day: day
    )
    @entriesArray << Entry.new(title,link,days)
end

Answer 2

uniq, , link ( find_by , , ), ( , ?) - , , )

, link , :

entries.each do |row|
        scrap = Scrap.create_with(title: row.css('h2>a').text, day: row.css('time').text).find_or_initialize_by(link: row.css('a')[0]['href'])
        @entriesArray << Entry.new(title,link,days) if scrap.new_record? && (scrap.save! if scrap.new_record?)
    end

( if , Entry, Entry, , if scrap.new_record? ... end

Answer 3

Scrap :

validates :title, uniqueness: true 
validates :link, uniqueness: true

.