mongodb批量处理

生铁刘老五 發表於 2019-10-10 15:37:00

mongodb支持批量插入。
1.使用Java mongodb api
查看源码com.mongodb.MongoCollectionImpl，有两个方法
<div class="cnblogs_code">
<pre>@Override
public void insertMany(final List<? extends TDocument> documents) {
 insertMany(documents, new InsertManyOptions());
}

@Override
public void insertMany(final List<? extends TDocument> documents, final InsertManyOptions options) {
 notNull("documents", documents);
 List<InsertRequest> requests = new ArrayList<InsertRequest>(documents.size());
 for (TDocument document : documents) {
 if (document == null) {
 throw new IllegalArgumentException("documents can not contain a null value");
 }
 if (getCodec() instanceof CollectibleCodec) {
 document = ((CollectibleCodec<TDocument>) getCodec()).generateIdIfAbsentFromDocument(document);
 }
 requests.add(new InsertRequest(documentToBsonDocument(document)));
 }
 executor.execute(new MixedBulkWriteOperation(namespace, requests, options.isOrdered(), writeConcern)
 .bypassDocumentValidation(options.getBypassDocumentValidation()));
}</pre>
</div>
<pre>insertMany(final List<? extends TDocument> documents) 默认使用 private boolean ordered = true;即有序插入。</pre>
<pre>insertMany(final List<? extends TDocument> documents, final InsertManyOptions options)调用者自己设置。 ordered属性有什么用？</pre>
<div class="cnblogs_code">
<pre> /**
* Gets whether the documents should be inserted in the order provided, stopping on the first failed insertion. The default is true.
* If false, the server will attempt to insert all the documents regardless of an failures.
*
* @return whether the the documents should be inserted in order
*/
public boolean isOrdered() {
 return ordered;
}</pre>
</div>
大概意思是：
true：按提供的顺序插入文档，并在首次插入失败时停止。（按顺序插入文档，遇到失败后停止。之前已经插入成功的不返回，失败及失败之后的不插入。）
false: 服务器将尝试插入所有文档，而不考虑失败。（只要能插入成功的，都存储进数据库）
 
2.spring-data-mongodb
org.springframework.data.mongodb.core.MongoTemplate
以下是该方法的源码和使用案例：
<div class="cnblogs_code">
<pre> /*
* (non-Javadoc)
* @see org.springframework.data.mongodb.core.ExecutableInsertOperation#bulkOps(org.springframework.data.mongodb.core.BulkMode, java.lang.String)
*/
public BulkOperations bulkOps(BulkMode bulkMode, String collectionName) {
 return bulkOps(bulkMode, null, collectionName);
}</pre>
</div>
<div class="cnblogs_code">
<pre>public int batchInsertStudents() { </pre>
<pre> BulkWriteResult result = null;</pre>
<pre> try {
 　　List<Student> documents = new ArrayList<>();
 　　String collectionName = "myTestCol";
 　　BulkOperations bulkOp = this.mongoTemplate.bulkOps(BulkMode.UNORDERED, collectionName);
 　　for(int i = 502610; i< 2000000; i++) {
 　　Student student = new Student();
 　　student.setId(String.valueOf(i));
 　　student.setAge(i);
 　　student.setGender("男");
 　　student.setName("李三"+ i);
 　　documents.add(student);
 　　}
 　　bulkOp.insert(documents);
 result = bulkOp.execute();
 }catch (DuplicateKeyException e) {
 System.out.println("**********" + e.getMessage());
　}
return result;
}</pre>
</div>
有两种模式：
<div class="cnblogs_code">
<pre> /**
* Mode for bulk operation.
**/
enum BulkMode {

 /** Perform bulk operations in sequence. The first error will cancel processing. */
 ORDERED,

 /** Perform bulk operations in parallel. Processing will continue on errors. */
 UNORDERED
};</pre>
</div>
与之前的order（true|false）相对应。
  
来源：https://www.cnblogs.com/muxi0407/p/11648383.html

頁: [1]

琼殿技术论坛's Archiver

mongodb批量处理